CS15 - Measuring Phishing Email Detection Accuracy Before and After a Student Checklist Intervention
SCURS Disciplines
Computer Sciences
Document Type
General Poster
Invited Presentation Choice
Not Applicable
Abstract
Phishing is still one of the easiest and most common ways for attackers to get into accounts and steal credentials, mostly because it targets peoples judgment more than it targets technical weaknesses. In this study, I'm testing a simple idea; does a short checklist actually help students spot phishing emails better than jsut going off instinct? The main question is whether a six-item phishing checklist increases detection accuracy compared to no guidance.
To test this, participants do a two-round email sorting activity using simulated emails that look like normal campus/work messages. In round 1, they'll label each email as either legitimate or phishing and rate how confident they are with their selection. After that, they get a quick checklist with practical things to look for. These items include the senders domain not matching, urgent or threatening language, link ext not matching the real destination, unexpected attachments, requests for login/payment info, and weird formatting or tone. Then in round 2 they sort a different set of emails that's designed to be similar in difficulty and topic.
The main results I'm tracking are accuracy, false positives, and false negatives. Ill also look at whether confidence changes and which checklist items people say helped them identify phishing emails the most. The analysis is a before-and-after comparison plus a summary of the most common mistakes people make.
This project is about treating phishing detection like a real measurable skill instead of just telling people to be careful. If the checklist improves accuracy, it can be supported as a cheap, simple training method that could work in student orientations, basic security training or helpdesk onboarding. It should show which phishing cues students miss the most, so future training can focus on true, researched weak spots.
Keywords
Phishing, Cybersecurity awareness, Social engineering, Email security, Human factors, Security training, Decision-making, Risk perceptoion
Start Date
10-4-2026 9:30 AM
Location
University Readiness Center Greatroom
End Date
10-4-2026 11:30 AM
CS15 - Measuring Phishing Email Detection Accuracy Before and After a Student Checklist Intervention
University Readiness Center Greatroom
Phishing is still one of the easiest and most common ways for attackers to get into accounts and steal credentials, mostly because it targets peoples judgment more than it targets technical weaknesses. In this study, I'm testing a simple idea; does a short checklist actually help students spot phishing emails better than jsut going off instinct? The main question is whether a six-item phishing checklist increases detection accuracy compared to no guidance.
To test this, participants do a two-round email sorting activity using simulated emails that look like normal campus/work messages. In round 1, they'll label each email as either legitimate or phishing and rate how confident they are with their selection. After that, they get a quick checklist with practical things to look for. These items include the senders domain not matching, urgent or threatening language, link ext not matching the real destination, unexpected attachments, requests for login/payment info, and weird formatting or tone. Then in round 2 they sort a different set of emails that's designed to be similar in difficulty and topic.
The main results I'm tracking are accuracy, false positives, and false negatives. Ill also look at whether confidence changes and which checklist items people say helped them identify phishing emails the most. The analysis is a before-and-after comparison plus a summary of the most common mistakes people make.
This project is about treating phishing detection like a real measurable skill instead of just telling people to be careful. If the checklist improves accuracy, it can be supported as a cheap, simple training method that could work in student orientations, basic security training or helpdesk onboarding. It should show which phishing cues students miss the most, so future training can focus on true, researched weak spots.