Self-regulated learning (SRL) is crucial for helping students attain high academic performance and achieve their learning objectives in the online learning context. However, learners often face challenges in properly applying SRL in online learning environments. Recent developments in artificial intelligence (AI) applications have shown promise in supporting learners’ self-regulation in online learning by measuring and augmenting SRL, but research in this area is still in its early stages. The purpose of this study is to explore students’ perceptions of the use of AI applications to support SRL and to identify the pedagogical and psychological aspects that they perceive as necessary for effective utilization of those AI applications. To explore this, a speed dating method using storyboards was employed as an exploratory design method. The study involved the development of 10 AI application storyboards to identify the phases and areas of SRL, and semi-structured interviews were conducted with 16 university students from various majors. The results indicated that learners perceived AI applications as useful for supporting metacognitive, cognitive, and behavioral regulation across different SRL areas, but not for regulating motivation. Next, regarding the use of AI applications to support SRL, learners requested consideration of three pedagogical and psychological aspects: learner identity, learner activeness, and learner position. The findings of this study offer practical implications for the design of AI applications in online learning, with the aim of supporting students’ SRL.
Fully online or blended courses taught via online video have become increasingly prevalent in higher education. However, unlike face-to-face lectures where instructors can support learners in regulating their own learning, online learning environments often provide learners with high levels of autonomy and low levels of instructor presence (Jansen et al., 2020). Consequently, learners’ self-regulated learning (SRL) becomes critical to their academic success (Kizilcec et al., 2017). Unfortunately, many learners struggle to self-regulate effectively in online learning (Greene & Azevedo, 2010; Jansen et al., 2020), resulting in a deficiency in utilizing their SRL during online learning (Winne & Baker, 2013). Therefore, it is essential to support learners’ SRL in online learning environments to enable them to achieve their learning goals effectively (Garcia et al., 2018).
External support using artificial intelligence (AI) could be a potential way of supporting learners’ successful SRL (Molenaar, 2022). However, supporting learners’ SRL is challenging as it comprises metacognitive, cognitive, behavioral and motivational processes (Lodge et al., 2019), and is heavily influenced by contextual and personal factors. Therefore, one-size-fits-all approaches to SRL support are ineffective (Roll et al., 2014). The use of AI to support SRL should have two objectives: to assess and interpret learners’ SRL behaviors in online learning environments (Noroozi et al., 2019; Roll & Winne, 2015) and to provide support that scaffolds these complex SRL processes. Various AI applications, such as AI plan organizers (Somasundaram et al., 2020), AI companions (Woolf et al., 2010), and AI agents (Goel & Polepeddi, 2016), have been developed to support learners’ SRL.
However, the learner's SRL can be influenced by factors such as motivation (Kizilcec et al., 2017) and self-efficacy (Zimmerman, 2013), as well as teachers' expertise and timely feedback. Given the significant impact of perceived teacher roles and learners' characteristics on the success of SRL (Jouhari et al., 2015), it is crucial to consider these aspects when effectively supporting SRL using AI applications. Zawacki-Richter and his colleagues (2019) highlighted the need for more research on the pedagogical and psychological considerations perceived by learners to effectively design AI applications in education. Similarly, Rosé and her colleagues (2019) emphasized the importance of providing interpretable and actionable insights into learners, rather than simply focusing on developing AI models that predict learners’ data more accurately. Additionally, it is important to comprehensively address potential issues learners may face when applying AI applications in real online learning environments, such as privacy violations, algorithmic biases, and surveillance (Seo et al., 2021a, 2021b).
To develop effective AI applications that support SRL in the long term, it is necessary to investigate students’ perspectives on such applications (Jivet et al., 2020). Students’ perceptions can serve as the basis for identifying pedagogical and psychological considerations in the development AI applications. The aim of this study was to identify how students perceive the use of AI applications in supporting SRL, and to explore the pedagogical and psychological aspects recognized importantly by learners when using AI applications to support SRL. To achieve the purpose of the study, we used a speed dating research method, a user experience design method that allows participants to interact with and experience various AI applications without any technical implementation process (Zimmerman & Forlizzi, 2017). The results of this study are expected to provide practical implications for the utilization and design of AI applications, aimed at supporting SRL in online learning environments.
The recent growth of online learning, including fully online or blended courses taught through online video, has brought changes to the learning environment for students and offered them more control over their learning (Jansen et al., 2020). However, to succeed in such an autonomous environment, learners must engage in productive SRL. Previous studies on SRL in online learning environments have shown that effective use of SRL strategies can lead to improvements in learners’ achievement. Moreover, studies have proposed specific strategies to promote learners’ adoption of SRL strategies in online learning (Kim & Hodges, 2012; Taub et al., 2014).
Table 1 summarizes a total of 10 representative SRL strategies that appear across three phases (i.e., forethought, performance, and reflection) and four areas (i.e., cognition, metacognition, motivation, and behavior) in online learning. Zimmerman’s (2000a2000b) three-phase cyclical model divides SRL into forethought, performance, and reflection phases, which are explained in the first column of Table 1. The forethought phase refers to the process that occurs before efforts are made to learn, such as setting goals or making plans. The performance phase refers to the process of monitoring whether the learning activities and strategies of learners meet the goals that were set during the first phase. The reflection phase refers to the process by which learners evaluate their own learning process and reflect on all the steps that they have taken to improve with regard to their subsequent learning session. These three phases of SRL involve different SRL strategies in each of the four areas mentioned (as presented in the second and third columns of Table 1). The area of cognition concerns the cognitive strategies used by learners during learning. Metacognition refers to learners’ ability to reflect upon, understand, and control their own learning. The area of motivation relates to the different motivational beliefs that individuals may have regarding their own learning abilities. Lastly, the behavior area reflects the general efforts that learners may make during the learning process.
We recruited research participants by a purposeful sampling method. To respond to the research question, the research participants were selected based on students who had experienced at least six months of fully online learning at universities and had diversity in academic achievement level, major background, gender, and grade. As a result, 16 university students (see Table 3) with backgrounds in 10 different majors in four different academic disciplines were selected, considering the characteristics of qualitative research in which the size of the sample is determined by informational considerations (Lincoln & Guba, 1985). No knowledge of AI applications was required on the part of participants, as we wanted participants to focus on the impact of AI applications on their SRL per se. Previous studies have shown that speed dating works well without any prior knowledge of or experience with AI applications (Luria et al., 2020; Zimmerman & Forlizzi, 2017). Each participant was reimbursed for their time with a gift card for coffee.
Table 3 Summary of the university student participants’ demographic informationWe conducted semistructured interviews with participants via a video conferencing platform (i.e., Zoom). We designed interview questions to understand how participants perceived the impact of the AI applications described in storyboards on their SRL (see Appendix A). First, to help students recall their online learning experiences, we asked the following question: “What was the most difficult part of learning online?” Subsequently, participants read each storyboard aloud and explained how they thought AI would affect their SRL in online learning. Specifically, we asked the following questions: “If this AI application were incorporated into your online learning, when and how would you use it?” “If this AI application is unlikely to help your SRL in your online learning, why is that the case?” and “What are some new things you would like to recommend or change with regard to this AI application? What are your concerns?” In addition, to obtain a holistic perspective on SRL in online learning, we asked participants to select AI applications that would be likely or unlikely to support SRL well. Note that to ensure that participants had the same or similar understanding of the AI applications described in the storyboards, if participants had any misconceptions about what AI could do for their SRL, we actively informed them. The entire interviews lasted approximately 45.3 min each (SD = 9.3 min), with 3–5 min spent sharing each storyboard and examining participant responses.
Each interview was audio recorded and transcribed for analysis. Analysis of the interview data followed the qualitative content analysis procedure (Glaser & Strauss, 1976; Schreier, 2012). First, all five authors repeatedly read the transcribed data and open-coded the contents that appeared to be meaningful in relation to the research question, identifying a total of 432 semantic codes. Subsequently, these semantic codes were clustered using axial coding to derive 46 final codes. With regard to this total of 46 final codes, Sect. "Learners’ perceptions on using AI applications to support SRL" presents 25 codes representing learners' perceptions of the use of AI applications to support SRL (RQ1). The remaining 21 codes were identified as seven themes and three factors related to the pedagogical and psychological aspects that affect learners’ use of AI applications (RQ2) based on selective coding, and these results are reported in Sect. "Pedagogical and psychological aspects for learners' use of AI applications". Each code and theme were determined to be mutually exclusive and distinct from each other. To ensure the validity of the data analysis, all authors participated in a total of seven instances of coordination and discussion until consensus was reached.
Table 4 summarizes the results of the analysis of responses obtained from the 16 participants on the use of AI applications to support SRL. The participants’ perceptions of how AI applications could be helpful for SRL were classified and organized based on the three learning phases (i.e., forethought, performance, and reflection) and four self-regulation areas (i.e., cognition, metacognition, motivation, and behavior). Most participants explained their intention to use AI applications in line with the SRL strategies that corresponded with the design intentions of the applications. Some students also reported being supported by other SRL strategies alongside the AI application design intentions, while others claimed that certain AI applications were not helpful for SRL. Quotations indicating the scenarios ("S") and participants ("P") were included while presenting their specific responses to each area of SRL.
Table 4 Learners’ perceptions of SRL support by AI applicationsSRL strategies during the forethought phase include planning, recognizing task values, and checking prior learning knowledge. All 16 participants responded that they would use the Plan Organizer (S01) to plan their learning. Students acknowledged the fact that the AI Companion (S06) would be effective in motivating them, especially when they needed to prepare for an exam or employment (P04, P05, P10, and P12). In addition, the companion is useful “because it provides a basis for changing the learning method” (P04) or modifying the learning plan (P02) by allowing students to recognize their deficiencies. However, some participants expressed the opinion that “university students are not motivated just by highlighting the possibility of career achievement because they are realistic” (P05 and P10). Learners used the Pre-question Generator (S02) to ensure that they had acquired prior knowledge (P01, P03, P04, P06, P09, P11, and P12) or to prepare for the next lesson (P10 and P13). However, some participants stated that they do not use this because it is either annoying to participate in the quiz (P04 and P07) or failed to reflect their grades (P15 and P16).
The core strategies associated with the performance phase include applying cognitive or motivational strategies, monitoring the learning process, and seeking help with difficulties. Participants noted their intent to utilize the Intelligent Suggestion to apply cognitive regulatory strategies such as enhancing their content understanding (P03, P05, P06, P07, P08, P09, P10, and P12) and implementing learning strategies (P01, P02, P03, P04, P05, P10, and P11). An additional advantage of promoting such an understanding of learning content is that “learners were less likely to procrastinate on their studies” (P09). Only two participants (P05 and P08) perceived that Virtual Human, designed to support motivational regulation, increases focus on learning. However, most participants noted that it is not helpful for students to focus (P03, P04, P05, P09, P10, and P11) or to interfere with learning (P02, P06, P07, P10, P12, P13, P14, and P15). All participants found AI Agent useful because it provides immediate answers to questions. Learners used AI Analytics to monitor their learning and establish learning strategies for cognitive regulation. However, most participants claimed that AI Analytics provides motivation for learning participation, and P04 and P13 suggest that it also increases confidence through comparison with other learners.
During the reflection phase, learners engaged in critical thinking and self-reflection by reviewing learning content and evaluating their own learning process and results. Core strategies such as reviewing, self-evaluation, and self-satisfaction were commonly applied. Participants reported that Adaptive Quiz (S09) helped them check how much they understood the learning contents (P01, P05, P06, P08, P10, P11, P12, and P13) enhanced their understanding of the learning content. In addition, P02 suggested that Adaptive Quiz contributed to learners’ feelings of achievement. However, P04 raised a problem in this context: "Solving questions that fit my cognitive level does not help me achieve the learning objectives of the course.” All participants mentioned that they would use Notelink to review what they did not know from their notes. Self-satisfaction was mainly mentioned when learners felt recognized, praised, and comfortable with their performance after using AI Reflection (S10). For example, P02 said, "Praise will improve not only the sense of achievement and willingness of learning but also self-esteem”. On the other hand, some students (P01, P05, and P07) did not intend to use AI Reflection because the praise they received was not reflected in their grades.
Overall, learners recognized that AI applications to support metacognitive, cognitive, and behavioral regulation in different areas of SRL are generally helpful, whereas the use of AI applications (S03 and S06) to support motivational regulation is not. AI Analytics (S05) was the most helpful AI application for learning, chosen by 10 out of 16 participants, while five participants chose AI Agents (S07). These findings confirm that learners find dashboards that objectively present their learning status or functions that automatically answer questions to be more useful. However, no participants selected Virtual Human (S03) or AI Companion (S06), which supported motivational regulation, as the most helpful AI applications for learning. This finding suggests that the use of AI applications to affect learners’ minds may have different effects for each learner. Furthermore, 13 learners selected Virtual Human, and three learners selected AI Reflections as the AI application they would use the least. From this point of view, AI applications must take a more detailed approach that takes into account the needs and characteristics of students to have an effect that is consistent with the intention of such applications to support learners' self-regulation.
Participants recognized that for learners to effectively utilize AI applications that support SRL, three major pedagogical and psychological aspects should be considered important: learner identity, learner activeness, and learner position. Each aspect emphasizes the multilayered characteristics of the learner based on their context, and depending on the extent to which the AI application takes these factors into account, the applications had both advantages and limitations with regard to the learners’ use of the AI application. Table 5 illustrates the aspects that were revealed based on the results of the analysis as well as the related list of themes and codes.
Table 5 Summary of three aspects for learners’ use of AI applications in supporting self-regulated learning
Learner identity is defined in terms of how an individual feels about himself or herself as a learner and the extent to which he or she describes himself or herself as a ‘learner’ (Lawson, 2014). Participants noted that their personal characteristics, such as their circumstances, learning styles, life patterns, and interests, can change continuously. For example, some learners claimed that "An individual's life pattern may change frequently due to part-time work outside of schoolwork” (P04) and "The learner’s level of interest in his or her major changes frequently” (S6, P01). These opinions were mainly expressed with regard to Plan Organizer (S01), Intelligent Suggestion (S04), or AI Companion (S06) because these scenarios predict future behaviors based on learners' historical data. Therefore, when it was felt that AI prescriptions did not sufficiently consider the various variables associated with learners, some learners regarded it as pressure to follow. Therefore, when it was felt that AI prescriptions did not sufficiently consider the various variables associated with learners, some learners regarded it as pressure to follow. If so, it is inevitable that learners will have doubts about whether their learning can be improved through AI. For example, P14 identified himself as a developing learner with ‘growth potential’ and suggested that it is important to ensure that AI applications do not result in limiting learner’s abilities by taking into consideration the learner’s identity.
Simultaneously, participants differentiated themselves from other fellow learners based on various characteristics, such as whether they were interested in celebrities (S03, P07) or whether they had decided on their career path to continue their studies (S06, P03, P04, P06, P13, P14, and P15). Participants viewed the usefulness of AI applications differently depending on whether each application was well suited to their personal characteristics in most scenarios. If the AI’s suggestions did not match the individual characteristics of the participants, they felt negative emotions such as “AI applications are interfering with me” (S06, P02) or “pressuring me” (S06, P08). P11 also expressed, “I am afraid that they will continue to compare me to my fellow students based on the same standards” in S05. In this regard, the participants suggested that "learners should be able to directly add or modify the application’s suggestions based on their personal information” (S01, P13 and P15).
Learners expressed their opinions on different levels of active patterns of action regarding their relationship with AI applications, ranging from AI dependence to learner agency. Learner agency is a key element in self-determined learning, which refers to a situation where learners take full responsibility for their own learning experience. This element is emphasized more frequently in online learning, which requires learners to have self-regulation ability (Agonács & Matos, 2021). The use of dependent AI in this context could be viewed as an experience that weakens learner agency the most. Participants were most concerned about the possibility of dependence on AI Agent (S07). This is because it was thought that the AI Agent would “interfere with helping creatively perform the tasks alone” (P02) and cause the possibility of “students solving the task in the same way” (P15). This means the concern “AI can impede learner’s cognitive efforts” (P14).
However, AI was also viewed as a useful way for learners to engage in active learning. We termed this phenomenon "learner-AI cooperation" in the sense that learners collaborate with AI while occupying a leading position. Time efficiency, just-in-time, and convenience are the features of the “learner-AI cooperation” strategy. Intelligent Suggestion (S04) has the effect of “reducing the time required for learning” (P01) and "preventing cramming learning” (P09) because it contains lecture content tailored to their learning style. The AI Agent (S07) is easy to use “when it is difficult to ask a professor questions late at night” (P03) and “when I want to check what I do not know right away” (P12). Some students noted that using AI Agent would be convenient when they were “ashamed to ask an instructor publicly using the LMS [learning management system]” (P13 and P14) or “confused about whether a question is too trivial to ask a teacher” (P07).
Meanwhile, some participants presented pedagogical alternatives to support learner agency. Focusing on five scenarios (S01, S04, S05, S09, and S10), they suggested that it would be better for AI applications to play the role of informing 'learning directions' rather than providing 'correct answers' to learning tasks (which was coded as ‘process oriented’). In the case of AI Reflection (S10), the AI “encourages growth in learner’s deficient areas” (P11 and P12) and "tell learners how to make up for deficiencies” (P12). Additionally, participants wanted their thoughts and needs to be reflected in AI data independently (which was coded as ‘content modification’). It is hoped that learners can control the database to be analyzed by AI by selecting “the topics they are interested in” (S02, P10) or “the problem-solving level that suits them” (S09, P12). These opinions suggest that supporting learners’ activeness is an important pedagogical and psychological aspect for students when using AI applications to support SRL.
Regarding the use of AI applications, the participants faced a situation in which they are both ‘independent learners’ and ‘dependent learners.’ Namely, they are students who engage in learning responsibly, but they are simultaneously subordinated to evaluations, credit requirements, and instructors. The ambivalence of the learner's position caused learners to perceive the educational usefulness of and concerns regarding AI application differently. First, as ‘independent learners,’ participants viewed AI applications as providing them with a variety of forms of cognitive support, such as helping them “identify core content” (S02) or “remember previously learned content for a long time” (S09). Furthermore, this usefulness eventually promoted learning efficiency, such as “reducing course learning time” (S04, P11), “reducing effort required to search for customized materials” (S04, P14), and “helping to prepare effectively for the test” (S09, P12). As independent learners, it is expected that participants will be able to check what they did well and what they lacked this semester and improve their ‘self-understanding’ through AI applications (S10, P02). They asked for specific feedback and advice on their learning status through the dashboard to improve positive motivation and self-esteem for learning (S10, P02, S05, P10, P11, P12, and P15).
On the other hand, as 'dependent learners,' participants focused mainly on the relationship between the use of AI applications and the exam or their grades. This means that even if learning information is provided according to the level of individual learners, the usefulness of tests or credits becomes the most important criterion for whether or not to use the information (P01, P03, P15, and P16). Participants recognized that quizzes or tests given under conditions unrelated to the final grade or credits were only a “psychological burden” (S02, P11) or “pressure” (S02, P10) to students. In other words, it was considered important in relation to “fairness” (P10) that different learning materials customized to learners should not affect the quality of the final evaluation. It is also a characteristic of dependent learners that participants wished to be able to choose whether to disclose (coded as selective disclosure) the results of the Quiz (S02) or AI Analytics (S05). Participants considered whether to use instructors rather than their own needs for AI applications and suggested that "learners should provide quiz questions that match the level of test questions presented by professors rather than their own learning level (S09, P07)." This fact is related to the university’s evaluation system where instructors hold the final decision-making authority. Accordingly, some participants expressed opinions that direct communication with the instructor would be more beneficial in terms of professionalism (S03, P05, S07, P01), indicating that their preference for instructor guidance over AI support as a factor affecting their learning motivation (coded as teacher preference). This means that learners’ individual learning should not be disconnected from the externally set achievement goals for learners to reach, and that customized support through AI applications should also have realistic usefulness for learners in this regard.
The study results indicated that learners use AI applications designed to support SRL strategies as intended. This finding aligns with Gracia et al.’s (2018) systematic literature review of e-learning tools, including AI applications, which demonstrated their effectiveness in supporting 14 self-regulated learning strategies proposed by Zimmerman (1989). However, some participants perceived that AI applications were not useful in supporting motivational regulation. It is important to note that learning is a complex process that involves emotional factors, such as motivation and context, extending beyond cognitive capacity. (Bates, et al., 2020). Online learning environments face challenges related to a decline in students' motivation, particularly for those who have not developed SRL skills (Karaoglan et al., 2018; Karaoglan Yilmaz & Yilmaz, 2021). Jones and Castellano (2018) suggest that SRL tutoring can help less able learners increase their motivation to engage in SRL practices. However, the study suggests that AI support may not effectively address this challenge. Learners expressed a preference for human support for motivational regulation due to the trust relationship built with instructors. This finding is consistent with Krüger and Wilson's (2022) study, suggesting that users may not follow AI recommendations without clear trust relationships. To address this challenge, future research is needed to understand the mechanisms that build trust in AI systems and to develop strategies to promote motivation in online learning environments.
Next, the results of this study indicated that the human factors affecting learners’ self-regulated learning (SRL) were perceived at a more sensitive and detailed level in the learning environment that utilizes AI applications, which are not directly visible to teachers or fellow learners. Leaners responded sensitively and tried to capture clues of human factors that were formally or informally influenced in relationships with instructors and fellow learners through AI support. Chen et al. (2014) have suggested that the success of AI applications heavily depends on how learners utilize them. Hence, to support SRL effectively through AI applications, it is crucial to design them by considering the pedagogical and psychological aspects of students using them, rather than just focusing on technical aspects. Firstly, when designing an AI application, it’s important to prioritize the learner’s potential progression, rather than relying on patterns from their past data. This is because elements of the learner's identity, including their circumstances, learning style, lifestyle, and interests, are subject to ongoing changes. For example, in ‘Plan Organizer,’ AI should be designed to offer suggestions and guidance to the learner regarding the action plans that he or she must take in the context of his or her circumstances, even if the plan is unfavorable based on the learner’s historical data. Secondly, for successful SRL support in AI application design, the degree of AI intervention should be adjusted according to the degree of the learner’s activeness. If the learner exhibits low activeness, it’s crucial for the AI application to take a more proactive role in aiding their SRL. Conversely, if the learner is highly active, the AI application should gradually diminish its intervention, facilitating the learner to undertake more self-determined learning. The AI’s role should be to bolster, not supplant, the learner’s SRL. This finding can be viewed as a practical design implication for realizing hybrid human-AI regulatory system, as described by Molenaar (2022). Lastly, when designing AI applications, researchers must carefully consider the positions of learners, instructors, and AI. Dependent learners, mainly prioritizing exams or grades, might sidestep AI assistance even if it improves their SRL, if they perceive that the AI application doesn’t contribute to grade improvement. Therefore, in the design phase of AI applications, instructors need to contemplate the influence of AI applications on the interaction between learner and instructor (Seo et al., 2021a, 2021b). They must then decide whether to utilize such applications merely as supplementary tools or as integral components embedded within the curriculum. Overall, for a more effective support of student SRL in online environments, it’s crucial consider pedagogical and psychological design aspects such as learner identity, learner activeness, and learner position during the design and deployment of AI applications.
In this study, we examined learners’ perceptions of the use of AI applications to support SRL, as well as the related the pedagogical and psychological aspects, through a speed dating activity with storyboards. The results showed that learners perceived that the use of AI applications to support metacognitive, cognitive, and behavioral regulation in the context of SRL was useful but also that the use of AI applications to support motivational regulation was not useful. Additionally, three pedagogical and psychological aspects (i.e., learner identity, learner activeness, and learner position) were found to be necessary for the development and utilization of AI applications to support learners’ SRL. Our proposed theoretical and practical implications with regard to the ability of AI applications to support the SRL of students in online learning could be used to realize human-AI symbiosis in education.
The authors would like to thank all students, educational experts, and AI experts for their great support and inspiration.
