How students use (and don’t use) ChatGPT.

The survey also found that there are preferences for ChatGPT use across majors and for the type of assignment completed. (See the ‘Student Rating of ChatGPT’ chart.)

Accounting majors relied on ChatGPT significantly less than non-accounting majors (Question 1: 2.8 vs. 3.4, respectively). Additionally, students reported much less reliance on ChatGPT for qualitative assignments compared to quantitative assignments (Question 1: 2.5 vs. 3.5, respectively). And there was a stronger reliance on class resources for qualitative assignments than for quantitative assignments (question 2: 4.9 vs. 4.0, respectively).

Student feedback shows that ChatGPT was used more for solving simple accounting calculations than for assignments that required deeper critical thinking skills.

Based on these results, it appears that students view ChatGPT as a better addition to, rather than a replacement for, traditional teaching materials. The following student comments reflect this sentiment:

“Every time I used ChatGPT, I compared the answers to notes and the textbook, and I found that there were usually small differences.”

“I’m very interested in AI, but prefer to use its responses to complement and assist my work, or to help me find a starting point when I’m at a loss.”

Experienced helpfulness of ChatGPT

Students generally reported that ChatGPT was somewhat helpful. They found it more useful for quantitative assignments than for qualitative assignments. Additionally, non-accounting majors found it more useful than accounting majors. This is highlighted by students’ willingness to use ChatGPT in non-accounting courses (Question 11: 4.6) versus other accounting courses (Question 10: 3.6).

Students had mixed feelings about ChatGPT’s helpfulness in finding correct answers (Question 6: 3.5) and specific formulas (Question 8: 3.1). They were just above neutral (Question 4: 4.2) regarding the perceived helpfulness of the technology in learning the material covered in the assignment. Instead, they reported that ChatGPT helped clarify understanding of the questions (Question 7: 4.7), provided detailed descriptions (Question 5: 5.1), and improved the quality of their responses to the assignments (Question 3: 4.6).

Key student comments reveal general thoughts on the usability of the technology:

“I mainly use ChatGPT to expand my thoughts and collect ideas.”

“I’ve been using ChatGPT as a tool when it comes to understanding concepts.”

Is using ChatGPT cheating?

Interestingly, students were generally neutral (Question 12: 4.0) on whether using ChatGPT was considered cheating. Also interesting is that non-accounting majors would be more willing than accounting majors to use it only if instructed to do so by their professors (Question 13: 5.4 vs. 4.9, respectively).

The following student comments provide ideas on whether using AI is cheating:

“I don’t see ChatGPT as a useful tool because it does the thinking for you, and therefore I see it as cheating.”

“Where exactly is the line between you learning something and being able to apply it later, and AI just doing the work for you?”

Trust (or lack of) in ChatGPT

In another finding, students reported moderate confidence in ChatGPT output (question 14: 3.7). However, this confidence varied significantly between non-accounting and accounting majors (Question 14: 4.1 vs. 3.2, respectively) and on quantitative vs. qualitative assignments (Question 14: 4.2 vs. 3.0, respectively).

A student who used ChatGPT to complete both a quantitative and qualitative assignment expressed skepticism, noting:

“After using it a few times for this class and Dr. Henderson’s class, I found that I trust myself more than ChatGPT.”

Despite this lack of confidence, regardless of major and assignment type completed, students reported higher levels of confidence in using ChatGPT for a future assignment (Question 15: 4.6).

The table below shows how the 129 Sam Houston State University students who used ChatGPT for one of three accounting classes in spring 2024 rated the generative AI tool. The results are broken down by assignment type.

Student review of ChatGPT