May is “AP Exam Season” in high schools. I have been preparing review material for the AP Computer Science Principles Exam and found that generative AI can be a helpful tool for coming up with practice questions. GenAI is not a magic wand able to produce an infinite number of usable questions, but it can assist a teacher with the process.
Below is an example of the prompt-and-response process that I engaged in using ChatGPT 4o.
Some observations:
- The prompt matters. When I first used ChatGPT to produce exam questions, I didn’t include the phrase “multiple choice questions with short scenarios” and the questions that were generated were very low-level recall style, for example: “Which is the correct definition of…”
- Generated responses are very stereotypical. After doing a couple questions around learning objectives dealing with extracting information from data, I found that ChatGPT’s themes for the scenarios are quite repetitive (temperature trends, music trends, fitness tracking).
- Using GenAI to write a question takes time. Each proposed question needs to be carefully considered, and then the deficiencies need to be articulated as a prompt.
- Educator expertise is needed. The quality and accuracy of the questions still depends on the prompter.
Prompt-Response Transcript:
The learning I want to assess is student understanding of AP CSP Essential Knowledge DAT-2.A.1: “Information is the collection of facts and patterns extracted from data.”
Prompt:
Write 2 multiple choice questions with short scenarios to assess understanding of: Information is the collection of facts and patterns extracted from data.
Response:
Of course! Here are 2 multiple choice questions with short scenarios to assess understanding that information is facts and patterns extracted from data:
1. Scenario: A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day.
Question: What is the information in this situation?
A. Each day's step count
B. The fact that her daily average is 7,500 steps
C. The number of steps she takes tomorrow
D. The brand of fitness tracker she uses
Correct Answer: B
2. Scenario: A school's attendance system collects data on when students are absent. At the end of the semester, the principal sees that absences are highest on Mondays.
Question: What pattern has been extracted as information?
A. Student names who were absent
B. The total number of school days in the semester
C. Absences are most common on Mondays
D. The number of teachers absent each day
Correct Answer: C
Would you also like a few challenge-style questions where students have to decide if something is "data" or "information"? That could be a fun extension!
I like the first question, but it feels too easy. I reword two of the choices and type in my version.
Prompt:
A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day. What is the information the app extracts from the data?
A. Each day's step count
B. Her daily average is 7,500 steps
C. Knowing that the number of steps she will take tomorrow will be 7.500
D. The number of days in the month
Response:
Here's your question cleaned up slightly for clarity, keeping your structure:
A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day. What is the information the app extracts from the data?
A. Each day's step count
B. Her daily average is 7,500 steps
C. Knowing that the number of steps she will take tomorrow will be 7,500
D. The number of days in the month
Correct Answer: B
Would you like a version with answer explanations too? (Helps if you're building a quiz or study guide!)
The third option (C) is written as an absolute, which is a common red flag.
Prompt:
Improve the wording of option C
Response:
Sure! Here's an improved version of option C to make it clearer and more accurate:
C. Predicting that she will take exactly 7,500 steps tomorrow
Would you also like a slightly tougher version where all the choices sound a little more similar, to really make students think carefully?
The suggestion is something that I usually do at the end (“Reword the choices to be the same length”). Later, I will deal with the quality of the suggested new wording.
Prompt:
Yes
Response:
Awesome! Here's a slightly tougher version of the question, where the answer choices sound more similar — making students think more carefully about what counts as data versus information:
A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day. Which of the following best describes the information the app extracts from the data?
A. The list of how many steps Mia took each individual day
B. The calculation that Mia’s average step count is 7,500 steps per day
C. The prediction that Mia will walk 7,500 steps tomorrow
D. The total number of days Mia’s steps were recorded
Correct Answer: B
Why this is trickier:
A is the raw data (individual daily steps).
B is the information (a fact extracted from analyzing the data).
C is a prediction, not information extracted directly from the existing data.
D is background context about the data collection, not the extracted pattern.
Would you also like me to create a "choose all that apply" version to push it even a bit further? (Those are great for checking deeper understanding!)
I am concerned that option C will be misunderstood as being based on a trend in the data, which would conflict with the intended correct answer. I expand the scenario and replace the option. I also clarify that the correct answer is the “best” example. Then, to double-check my changes, I re-prompt with my edits to see what the feedback is.
Prompt:
A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day and lists exercise activities to try next month. Which of the following is the best example of information the app extracts from data?
A. The list of how many steps Mia took each individual day
B. The calculation that Mia’s average is 7,500 steps per day
C. The activity ideas for next month
D. The number of days Mia’s steps were recorded
Response:
Here’s your question polished and ready, with slight phrasing tweaks for clarity:
A fitness app records how many steps Mia takes each day for a month. Afterward, it shows her that she averages 7,500 steps per day and lists exercise activities to try next month. Which of the following is the best example of information the app extracts from the data?
A. The list of how many steps Mia took each individual day
B. The calculation that Mia’s average is 7,500 steps per day
C. The activity ideas for next month
D. The number of days Mia’s steps were recorded
Correct Answer: B
Quick reasoning:
A = raw data
B = extracted information (summary fact or pattern)
C = advice not necessarily extracted from data
D = metadata about the dataset (context, not extracted pattern)
Would you also want a version where students have to explain why it's B (short explanation question)? Those really boost understanding!
I like ChatGPT’s feedback. I am satisfied with the quality. Question is done.
Teaching is STEM 
Leave a comment