Validitty of ICARE-Modification Model of Improving Students’ Critical Thinking Skills

Introduction

he Fourth Industrial Revolution has influenced the way people live, including how they interact and how education systems operate digitally ([23]; [12]; [25]). In this era, individuals face challenges related to the rapid development of information, which forms the foundation of human life. Information and technology are advancing very quickly, can be accessed anytime, and communication can occur at any moment (Rafzan et al., 2020; [24]). As technology continues to evolve, the term Industrial Revolution 5.0 has emerged. This concept is still under development and debate, but it generally refers to technological advancements that further enhance automation and digitalization in industry and production sectors. This concept focuses on the integration of technology and humans, as well as the need to develop systems that are more adaptive and responsive to changes in the production environment. Industrial Revolution 5.0 places greater emphasis on integrating advanced technologies such as Artificial Intelligence (AI), the Internet of Things (IoT), and robotics with human expertise and innovation to promote the development of production systems that are more efficient, flexible, sustainable, and capable of improving human well-being, particularly in higher education. Based on the explanation above, it is evident that the development of information technology is progressing very rapidly, indirectly encouraging the emergence of various innovative learning models across different fields. The emergence of new digital-based learning innovations is one way to overcome existing gaps and serves as a solution to various limitations of traditional learning methods. Learning methods packaged in digital formats can encourage more creative and engaging learning experiences in every field. One of the fields significantly affected by technological development is education. Fundamentally, education is a process of communication and information transfer from educators to students, containing educational content. It involves several key elements: educators as sources of information, media as tools for presenting ideas, concepts, and educational materials, and students themselves (Azrina & Latifah, 2020). As some of these elements have been influenced by information technology, this has led to the emergence of the concept of e-learning.

Moodle-based e-learning provides features that support the learning process; however, it needs to be combined with a learning method that has clear procedural steps. One relevant and feasible method to be integrated is the ICARE model (Introduction, Connection, Application, Reflection, and Extension). This learning model consists of five stages that guide students to learn independently and construct their own knowledge. In this study, the ICARE model is modified by incorporating, at the Reflection stage, an indicator that requires students to reconstruct explanations based on their reflections in order to identify assumptions related to problem-solving.

Furthermore, the ICARE-M (Introduction, Connection, Application, Extension, and Reflection)–Modification learning model can be designed with the assistance of Artificial Intelligence (AI) and implemented in e-learning through the Moodle application. This method is prepared to address the challenges of rapid technological advancement. Artificial Intelligence (AI) greatly assists in designing e-learning through Moodle using the ICARE-M (Introduction, Connection, Application, and Reflection)–Modification model. AI facilitates personalization by determining students’ levels of understanding, learning styles, and learning preferences. Course materials can then be tailored to individual student needs, thereby providing a more effective and productive learning experience.

AI also plays a significant role in helping students complete assignments and projects by offering useful tips, suggestions, and feedback. With AI tools such as ChatGPT-4o, students can receive quick and specific feedback, enabling them to understand their strengths and weaknesses in the learning process. By utilizing AI, educational institutions can collect and analyze large-scale data related to students and learning processes. This data can be used to identify students’ critical thinking skills, particularly in constructing extended explanations. For example, such data can be used to predict student performance and success, as well as to identify factors influencing student retention [7]. The ICARE-Modification learning model implemented in e-learning has been shown to improve students’ critical thinking skills [11]. Critical thinking has become a crucial issue in education to address the challenges of the Industrial Revolution 4.0 era ([8]; Ridho, Wadani, & Saptono, 2018; [11]; [14]; Theori et al., 2020). Education must respond to and anticipate challenges arising from science- and technology-based developments. Higher education institutions need to prepare high-quality graduates who possess critical thinking skills to compete globally in the era of Industry 5.0 ([15]; Azmi et al., 2018; [20]; [1]; Živković, 2016).

Education must also optimize its efforts and implement more effective interventions to prepare graduates with strong critical thinking skills ([5]; [26]; [2]).

According to Ennis (1993), critical thinking consists of the following abilities: (1) Providing simple explanations, with indicators including focusing questions, analyzing arguments, and asking and answering clarification questions. (2) Building basic skills, with indicators including assessing the credibility of sources, observing, and considering observation results. (3) Drawing conclusions, with indicators including making and evaluating deductions, making and evaluating inductions, and making and considering value judgments. (4) Providing further explanations, with indicators including defining terms, considering definitions, and identifying assumptions. (4) Strategies and tactics, with indicators including determining actions and interacting with others.

To meet the standards of critical thinking skills among biology education students, it is essential to validate the ICARE-Modification model book as well as supporting instructional materials, such as semester lesson plans, plant ecology materials, and Higher-Order Thinking Skills (HOTS) test items designed to enhance students’ critical thinking skills. Based on the background described above, the research problem of this study is:How feasible is the lecture design of the ICARE-M (Introduction, Connection, Application, Reflection, and Extension)–Modification learning model assisted by Artificial Intelligence (AI)? This research problem is further elaborated as follows: How valid is the lecture design of the ICARE-M (Introduction, Connection, Application, Reflection, and Extension)–Modification learning model assisted by Artificial Intelligence (AI) in improving students’ critical thinking skills through e-learning in terms of content and construct validity?

Research Method

This type of research is classified as Research and Development (R&D) adapted from the Plomp Model development design [18]. The development of the ICARE-Modification model is at least valid, practical, and effective [19]. Tessmer (1993) explains that the quality of research and development products is effective, efficient, motivating for users, easy to use, and affordable. The validity of the ICARE-Modification model is designed logically and consistently based on the state of the art of knowledge. The ICARE-Modification model is said to be valid if it meets content validity and construct validity. The development product is said to be practical if it is easy for teachers to implement. The ICARE-Modification model is said effective when it is able to improving students’ critical thinking skills. The purpose of this educational development research is to create specific learning models and assess the effectiveness of these model. The educational development research stages of the Plomp model consist of five stages: 1) Preliminary research, 2) Prototype design and construction), 3) Validation, 4) Assessment and 5) Implementation. This study presents the results of the validation stage. The validation stage was carried out by three experts in the fields of pedagogy, science content and assessment to validate the ICARE-Modification learning model book, learning tools and research instruments. These validators possess extensive experience in their fields and hold the highest educational qualifications, from doctoral programs to professorships, as experts in their fields. Validation instruments are used to determine the suitability and quality of the product on a scale of 1-5. The validity of the ICARE-Modification model product development results must meet at least the minimum standard criteria of 0.50 ≤ V < 0.75, categorized as valid. Differences arising from the validation process are then discussed in a group discussion forum to reach a consensus. The steps of the research method are interpreted in Figure 1.

The ICARE-Modification learning model and its supporting tools have been validated by three experts in the fields of sains, learning material and assessment. The validation data were then analyzed using aiken’s validity coefficient (V) with the following formula.

V = ^s/ n(c-1)

S = R - Lo

Description:

V = Aiken Index

S = score given by the assessor minus the lowest score in the category

R = Score given by the assessor

Validity of ICARE-Modification Model to improving critical thinking studens’

Lo = Lowest assessment score

C = Highest assessment score

n = Number of validators (assessors).

The validation results were then analyzed descriptively and adjusted according to the model validation categorization criteria as shown in Table 1.

The validity of the developed product results meets the minimum standard criteria in the interval 0.50 ≤ V < 0.75 with the valid category. The reliability of the model validation instrument and supporting tools for the ICARE-Modification learning model was determined by calculating the level of agreement between observers. Validation results are reliable if the reliability value ≥ 75% (Borich, 2017).

Result And Discussion

Results

The content validity of the ICARE-Modification learning model includes the feasibility of the substance content, the feasibility of the ICARE-Modification model design, the feasibility of E-Learning implementation, and the feasibility in terms of benefits and practicality. The validation data obtained from three experts in their respective fields were then analyzed using Aiken’s Validity Coefficient formula [3], and the reliability level was measured using the Percentage of Agreement statistical analysis (Borich, 2017). The results of the content validity measurement can be seen in Table 2.

V= Aiken’s validity score; R=Coefisient inter-observer agreement

The validation of HOTS (Higher Order Thinking Skills) questions was conducted to determine the feasibility of the HOTS items in improving students’ critical thinking skills. The results of the study can be seen in Table 3

V= Aiken’s validity score; R=Coefisient inter-observer agreement

The learning tools supporting the implementation of the ICARE-Modification model development have also been validated. Content and construct validation were conducted to measure the suitability and consistency of the supporting learning tools in improving students’ critical thinking skills. The supporting tools for the development of the ICARE-Modification model include the Semester Learning Plan, e-learning platform, plant ecology materials, and HOTS questions oriented toward students’ critical thinking skills. Three validators assessed the validity of these supporting tools using validation instruments provided by the researchers. The findings of the content and construct validation are presented in Table 4

Discussion

The ICARE-M (Introduction, Connection, Application, Reflection, Extension)-Modification model is considered valid and consistent. This is based on the results of the ICARE-M model validation conducted by three experts in their respective fields, including experts in pedagogy, science content, e-learning, and HOTS. The assessment results from the three validators are presented in Tables 1, 2, and 3. The content validation of the developed ICARE-M learning model has proven that it is valid and can be implemented in learning with minor revisions. The ICARE-M model fulfills six components, namely syntax, social system, principles of reaction, support system, instructional impact and nurturant effects, as well as a supportive learning environment. The ICARE-Modification learning model not only supports the achievement of conceptual understanding but also encourages students to become reflective and critical learners [4]. The combination of concrete experience, personal connection, practical application, reflection, and further exploration makes ICARE a highly relevant model for developing students’ critical thinking in 21st-century learning, as encouraging reflection, idea exploration, and open dialogue is highly effective in shaping critical thinkers (Brookfield, 2012). It is expected that enriching scientific content with conceptual knowledge related to plant ecology will further enhance students’ critical thinking skills, particularly in the indicators of questioning ability, analyzing observational results, and making assumptions related to solving environmental problems in their surroundings [6]. The development of the ICARE-Modification model does not only focus on students’ pedagogical competence but also on fostering students who are sensitive and wise in making decisions to solve environmental problems in their living environment. Therefore, the ICARE-Modification model can serve as one of the learning model variations that prioritizes the achievement of the goals of Education for Sustainable Development (ESD) [9].

The average content validity score of the ICARE-Modification model is 0.92, categorized as very valid, with a reliability level of 93.95%. The needs analysis of the ICARE-Modification model is considered highly relevant to the demands of 21st-century education, particularly in addressing environmental issues and enhancing students’ critical thinking skills. Students can engage in individual learning, group work, and direct field observations [10]. The steps of the ICARE-Modification learning model each play a distinct role in improving students’ critical thinking skills through HOTS-based questions (Jacob, S. M., 2012). The third step (Application) takes the longest time because it represents the core of the ICARE-Modification learning model. In this phase, students are required to conduct experiments/observations and work on HOTS questions or apply their knowledge in real-world contexts [21]. Activities in the Application step include individual and group projects designed to train students to think critically. Students integrate knowledge content with their surrounding environment, creating opportunities to ask critical questions and make assumptions based on discussion results and analysis of observational findings. In this step, students move from knowing to doing and reasoning, making learning active, contextual, and meaningful. This is supported by Kolb’s concept (1984) that learning is not sufficient by merely knowing, but must involve doing and reflecting (reasoning). This statement is also aligned with Jean Piaget’s constructivist theory, which emphasizes that meaningful learning occurs when students engage in exploration and independent reasoning.

The sequence of the ICARE-Modification model begins with the Introduction stage, which provides initial information related to the material to be taught, such as videos of environmental damage caused by human activities, websites containing news about environmental degradation, and HOTS questions to measure students’ critical thinking skills. At this initial stage, HOTS questions are given along with stimuli to trigger students’ curiosity. Interesting phenomena need to be presented in provocative problem situations and are expected to arouse curiosity through participation in the form of questioning (Slavin, 2018). When students possess strong curiosity, critical questions will emerge in order to obtain accurate answers. Students are trained to formulate critical questions (prompts) in an AI-assisted chatbox (ChatGPT) to obtain accurate and scientific sources of information. Students can analyze observational data displayed in the e-learning platform, solve existing problems, and formulate possible assumptions as solutions to environmental issues.

The second step, Connection, focuses on linking students’ prior knowledge with their previous learning experiences. In this stage, the lecturer presents HOTS-based guiding questions related to the material, then explores students’ misconceptions to activate their prior knowledge. This process fosters students’ curiosity, raises awareness of the relevance of the material to real-life situations, prepares their cognitive readiness to receive new concepts, and initiates the process of critical thinking. Students’ questioning ability most initially and dominantly emerges at this stage, as this phase is designed to build curiosity through the presentation of contextual phenomena. Likewise, the ability to make assumptions first and most prominently appears in the Connection stage, because this phase aims to activate prior knowledge and construct preliminary assumptions about the observed phenomena. However, the quality of these assumptions develops and is systematically tested in the Application stage, and is later re-evaluated in the Reflection stage.

The Application stage is the phase in which students apply the concepts they have learned to analyze and solve contextual problems. Through data processing activities and evidence-based discussions with group members, students not only understand concepts theoretically but are also able to use them in real-life situations. This stage promotes the development of higher-order thinking skills, particularly the ability to analyze observational results. The ability to analyze observational findings develops most strongly at this stage because students not only observe but also interpret data, relate it to scientific concepts, and draw evidence-based conclusions. The Reflection stage then strengthens the quality of analysis through evaluation and reinterpretation of findings, as well as refining assumptions based on data, making learning more active, contextual, and meaningful.

The Reflection stage is an essential step in reinforcing students’ knowledge construction through self-evaluation and meaningful interpretation of learning experiences. At this stage, students reflect on their conceptual understanding, identify difficulties, and formulate conclusions based on the results of discussions and investigations. The reflection process not only deepens conceptual understanding but also develops metacognitive awareness, thereby making learning more meaningful and sustainable.

The Extension stage is a phase of learning expansion aimed at encouraging students to transfer and generalize concepts into broader contexts. Through enrichment tasks and independent exploration, students are trained to think flexibly and adaptively, as well as to develop innovative solutions based on the scientific concepts they have learned. Thus, learning becomes continuous and is not limited to a single problem situation.

Based on Table 4, the validity of the supporting components of the ICARE-Modification model—including the semester lesson plan, E-Learning platform, Plant Ecology textbook, and HOTS questions—obtained an average score of 0.90 (valid category), with a reliability value of 91% (reliable category). The level of agreement exceeded 75%. It can therefore be concluded that the validity of the critical thinking skill indicators is considered valid with very good criteria. The content validity of the ICARE-Modification model development presented in the model book also meets very good criteria. This means that the ICARE-Modification model design described in the model book is categorized as highly feasible and consistent.

Learning to improve students’ critical thinking skills—particularly the indicators of questioning ability, the ability to analyze observational results, and the ability to make assumptions—requires a specific instructional design, namely: (1) Learning aimed at enhancing students’ critical thinking skills, especially in the areas of questioning, analyzing observational results, and making assumptions, requires a structured and systematic instructional design. This design should position students as active subjects through the presentation of contextual problems, observation activities, argumentative discussions, and evidence-based data analysis. (2) Questioning ability can be developed by presenting authentic phenomena and guiding questions that stimulate curiosity. The ability to analyze observational results is strengthened through data processing and interpretation activities linked to scientific concepts. Meanwhile, the ability to make assumptions develops when students are encouraged to propose conjectures or hypotheses based on prior knowledge and observed phenomena, and then test them through investigative processes. Thus, learning that is contextually designed, problem-based, and oriented toward investigative activities will be more effective in fostering these critical thinking indicators.

Conclusion

Fundamental Findings: The developed ICARE-Modification learning model has been proven to be valid in terms of content and construct and effective in improving the critical thinking skills of undergraduate students in Biology Education, Faculty of Teacher Training and Education, Mulawarman University. Implications: The ICARE-Modification learning model is designed based on constructivist theory, which positions students as active subjects in constructing knowledge through the inquiry process. Through the stages of Connection, Application, and Reflection, students are encouraged to ask questions, make assumptions, analyze observational data, and reflect on their thinking processes. This approach is aligned with the principles of inquiry learning, which are effective in developing critical thinking skills. Limitations: Students have limitations in formulating critical questions, particularly in designing group project objectives and determining appropriate assumptions to solve environmental problems. Future Research: The validity of the ICARE-Modification learning model needs to be continuously examined to ensure its effectiveness in classroom implementation across various topics and through the integration of technology in learning.

Acknowledgements

The author would like to express sincere gratitude to colleagues, observers, the Head of the Study Program, and the Dean of the Faculty of Teacher Training and Education, Biology Education, FKIP Mulawarman University, for their support and facilities during the implementation of this research.