Evaluating Effectiveness of Mathematical Training of Students within Electronic Educational Environment

(a) Mari State University, 424000, Yoshkar-Ola (Russia),1 Lenina Sq., toktarova@yandex.ru (b) Mari State University, 424000, Yoshkar-Ola (Russia),1 Lenina Sq., svetfed65@rambler.ru (c) Volga State Academy of Physical Culture, Sports and Tourism, 420000, Kazan (Russia), 35 Universiade Village St., olca.1970@rambler.ru (d) L.N. Gumilyov Eurasian National University, 010000, Nur-Sultan (Kazakhstan), 2 Satpaeva St., zarinchen@mail.ru


Introduction
The effectiveness of education is a fundamental criterion for the quality of the educational process in both traditional and innovative learning environment. At the same time, it seems to be fair to determine the evaluation of the effectiveness of education as a teaching system that contributes to obtaining information about the learning outcomes in the form of knowledge, skills, personal qualities and abilities acquired by students during this process (Zhigalev, 2007). Considering the categories of quality and effectiveness, it should be emphasized that effectiveness i s interpreted as the degree of achieving the goal at certain costs; or effectiveness can also be defined as quality (achievement of the goal) taking into account costs. At the same time, the main didactic tool to guarantee the effectiveness and quality of education is evaluation, since it provides feedback in the educational process, and is a confirmation that the quality requirements are met (or not met) (Zhigalev, 2007).
The effectiveness of education can be viewed from two perspectives: first, as the degree of compliance of the educational system with the established requirements; second, as the progress of the educational process that meets the needs of social customers, society and the state. At the same time, the effectiveness from the point of view of the quality of education is a multidimensional phenomenon and includes such phenomena as: the quality of educational programmes; the quality of the teaching staff; the quality of teaching; the quality of students, pupils, and high school graduates; the quality of material and technical base as well as electronic educational environment of the institution; the quality of management system; the quality of research, etc.

Purpose and objectives of study
The purpose of our research is to describe and justify the evaluation of the effectiveness of mathematical training of students within the electronic educational environment of the high educational institution.
The effectiveness of mathematical training of students is understood as a certain level of achieving the goals of learning mathematics and the degree of compliance of the process and the result of mathematical training of students with personal expectations and needs of society in accordance with the following parameters: − maturity of internal motivation to obtain and use mathematical educational information (availability of tools in the electronic educational environment to create student's needs and aspirations for learning and applying mathematical knowledge in professional activities); − volume, completeness and consistency of mathematical knowledge and skills (mathematical knowledge, skills and abilities should be presented in the electronic educational environment not in the form of scattered concepts and facts, but as a set of interrelated elements); − ability of students to individual search and processing of mathematical educational information (deep acquisition of mathematical knowledge is achieved only by individual analysis of new and complete information).

Literature review
Vast array of information has been accumulated in the theory and practice of pedagogical research, revealing the scientific foundations and experience in evaluating effectiveness of mathematical training of students in higher educational institutions.
When shifting during the process of mathematical training to the electronic educational environment, ensuring the high quality of training is becoming extremely important. Factors that influence the quality of e-learning within the EEE can be divided into external (state, social, demographic, financial) and internal (University policy in the field of e-learning, the quality of EEE arrangement and content, the level of competence of teachers and students in the field of information technologies, etc.) (Standards, 2008). Talyzina's (1984) point of view is of great interest as she notes the justification for the use of automated systems in training only if it leads to an increase in the effectiveness of training for at least one of the 2584Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-2020 following criteria: increasing the motivational and emotional feature of training; improving the quality of training; reducing the time spent by the student and the teacher for the subject; or reducing the expenditures for training.
Requirements and general approach to management and quality assurance in relation to e-learning are described in GOST R53625-2009R53625- (GOST, 2011. The main provisions of the standard are applicable to various forms of e-learning arrangement (mobile, network, autonomous, mixed, joint, etc.) and types of distance education technologies. The lifecycle processes for e-learning include the following stages: need analysis → structure analysis → concept/project → development/implementation → learning process → evaluation/ optimization. Each process contains a description of sub-processes / sub-aspects, a goal, a method, a result, actors, metrics / criteria, regulatory documents, and abstracts. The approach to determining the quality of e-learning in accordance with the standard is based on the development of detailed descriptions of each process and sub-process.
An important role in ensuring the effectiveness and quality of e-learning is given to the capabilities, means, and conditions of electronic educational environment functioning.

Methodology
The most important aspect of evaluating the quality of education is the choice of evaluation methods to measure learning outcomes at different stages, taking into account competency-based, taxonomic and qualimetric approaches.
The competency-based approach is based on the multidimensional and multi-structural characteristics of the quality of students' training, is focused on the formation level of competencies defined in the FSES HE. The evaluation of learning outcomes in terms of competencies determines what a student should know, be able to do, understand, and have skills upon completion of training. For this purpose, there are special created databases of evaluation tasks. To have a more complete picture, there are comprehensive developed competency passports consisting of a list of competencies that must be formed during studies from the first year to graduation. At the same time, to improve training effectiveness it is necessary to establish the students' achievements at each stage, to monitor the level of formation of their competencies, data collection and analysis at the quantitative and qualitative levels (Fomin, 2013). In relation to teaching mathematics, the requirements of modern educational standards also include a list of students' competencies formed during the process of mathematical training. At the same time, the system of professional standard tasks in mathematics should be designed so that it is possible to evaluate the development of the defined competencies. In addition, Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-20202585 a competence matrix should be built and presented in the EEE. It shall include a clear list of subjects, which may contribute to the formation of the necessary competencies.
The next approach, which should be taken into account along with the competency-based one is taxonomic which allows to determine the levels of educational goals and learning outcomes, to structure the system of evaluating tasks. All taxonomic descriptions that determine training effectiveness and quality are characterized by a gradual complication of cognitive activity of students. For example, Bloom et al. (1956) describe the levels of training or successive cognitive categories and the degree of information acquisition.
This taxonomy is presented in a structured form accessible for practical application, it is convenient for evaluating the quality of teaching mathematics in the conditions of electronic educational environment.
With a focus on the process of acquiring mathematical knowledge, the taxonomy will include the following levels: − the level of analysis: the ability to break the material into components in order to identify a specific structure (Choshanov, 2011): the analysis of elements is the ability to structure the material and isolate parts of the whole; the analysis of relationships is establishing links between elements; the analysis of the principles of organizing the whole is the systematization of elements, highlighting hidden assumptions, making distinctions between facts and consequences; − the level of synthesis: the formation of creativity skills: synthesis of ideas as a search for ideas, solutions, problems; synthesis of procedures is the development of a plan of action and the sequence of operations for solving the problem; synthesis of a structure is building functions, sets, and creating a diagram to organize information, etc.; 2586Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-2020 − the level of assessment: the formation of diagnostic skills that involve evaluating the material (compliance with the intended goal, reasonableness, logic, constructiveness, etc.).
At each of the six designated levels the student performs certain activities in accordance with the specific requirements for assessing the effectiveness and progress in learning mathematics.
The qualimetric approach is aimed at improving the objectivity of evaluating the level of students' knowledge; it allows to conduct statistical analysis of the results achieved and adjust the learning process (Clarin, 1994). In accordance with it, the evaluation is based on the qualimetric methodology (direction connected with quantitative description of the quality of subjects or processes), where the goals and the learning outcomes (including the level of formation of competencies) are presented in measurable values.
Each indicator characterizing the level of competency formation is given a numerical value. The results are measured by points, scaled, and analyzed by means of statistical and mathematical analysis.
A striking example of the qualimetric approach to evaluating the quality of teaching mathematics in the EEE conditions is the calculation of a single quality index (Lobanov, 2001). The quality index is a Undoubtedly, all the presented approaches will be useful in evaluating the effectiveness and quality of mathematical training within the electronic educational environment. But for a more complete result, it is necessary to use a set of different approaches, complementing them with appropriate e-learning methods.

Results
In order to ensure a single character of internal and external control and monitoring of training, we have developed and implemented the diagnostic toolkit for evaluating the effectiveness of mathematical training of students in the conditions of the University EEE (Fig. 1). -subject content (target orientation of mathematical material; applied orientation of teaching mathematics; presence of key tasks; variety of types of mathematical tasks; provision of computer mathematical modeling tools); 2588Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-2020 -technologies (applied technologies of the course; technologies of educational material delivery; technologies used for interacting of the subjects of training; completeness of the use of existing and new technological opportunities and resources); -adaptation (availability of input test to determine students' characteristics and needs, their initial level of preparedness; provision of a learning algorithm in accordance with the individual characteristics and preferences of a student; learning management; multi-level learning material; psychological comfort (focus on different types and styles of thinking); availability of didactic material and resources to people with disabilities); 3) software and hardware (functionality of tools and resources of the environment to meet the requirements of learning technologies; provision of the educational process with the necessary software; duration of downloading the electronic didactic materials and working speed of resources; provision of a possibility of multi-media preparation and presentation of educational material; freedom of access to information resources; possibility of distant learning; availability of an adaptive course on the formation of information competency of students; compliance and support of domestic and international standards; availability of a mobile version of the content of the electronic educational environment; ease of course navigation; easy access to courses and services; user-friendly interface); 4) communicative (variety and convenience of forms of pedagogical interaction of subjects of the educational process; structure and character of the dialogue; feedback with the teacher; support of students; possibility to perform group tasks); 5) personnel (qualification of a teacher / tutor; level of proficiency of a teacher / tutor in information and communication technologies; degree of availability of teachers; availability of technical and methodological support services for training).
For a more objective evaluation of the effectiveness of mathematical training of students in the conditions of electronic educational environment of the University, each criterion of the group is described by specific indicators that can be measured and evaluated in the future. For example, the criterion "Applied orientation of teaching mathematics" includes such indicators as: -selection and provision of educational material in mathematics in accordance with student's major; -availability of examples for professionally significant motivation of students; Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-20202589 -orientation of the training content to the study of mathematical theory in the process of solving tasks; -variety and correspondence of mathematical tasks to the future professional area of the student.

Discussions
To rank the selected groups of criteria / the criteria themselves in the group / parameters corresponding to the criteria in the developed diagnostic toolkit, we used the Saati hierarchy analysis method (Saati et al., 2008), which is a mathematical tool for a systematic approach to solving decision-making problems. The essence of the method is to determine the eigenvector of the judgment matrix A = (aij), (i, j = 1...n) with the highest eigenvalue based on a pair comparison of the studied parameters. A = where aij = α, then aji= 1/ α, if α ≠0; aij = 1 if i=j.
Construction of judgments αij about pairs (Ci, Cj), where C1, C2, ... Cn is a set of groups/ criteria / parameters for pairwise comparison of characteristics in accordance with a normalized evaluation intensity scale from 1 to 9, where the evaluations have the following meaning: 0 -objects are incomparable, 1 -equal importance; 3 -moderate superiority, 5 -significant superiority, 7 -strong superiority, 9 -absolute superiority, in intermediate cases there are even scores: 2, 4, 6, 8, when choosing between two adjacent odd numbers causes difficulty.
Evaluation of the eigenvector values of the matrix based on a pair comparison of the studied parameters provides ordering the priorities of the evaluated characteristics in the group of research parameters. The task is reduced to finding the maximum value of the eigenvector of the judgment matrix λmax, which is used to evaluate the consistency of expert judgments. The deviation from consistency is more often taken into account in practice, which is expressed by a value called the consistency index: where is the maximum eigenvalue of the judgment vector, n is the number of objects (or activities), and if the judgment is considered satisfactory.
Using the Saati method to obtain weight coefficients of objects in the judgment matrix allows to determine the contribution of a certain group / criterion / parameter in defining the integral indicators for different levels of the hierarchy.
In general, the dependence of the evaluation of the effectiveness of mathematical training within the conditions of the University EEE on any group of criteria can be represented as (Krasilnikova, 2009): , where is the effectiveness of the previous level of the hierarchy determined by the contribution of the criteria of the i-th level of the hierarchy; are the coefficients of the ith level of the hierarchy; is the current parameter of the considered criterion; n is the number of criteria/parameters for the hierarchy level under consideration.
The approach to building a dependency is performed from top to bottom, while the application of the obtained mathematical dependencies in the evaluating the effectiveness is performed in the reverse order from bottom to top, starting from the last level of the hierarchy where the parameters to be measured should be located. This level of hierarchy evaluates the contribution of 8 groups of criteria to the overall effectiveness of mathematical training in the conditions of the University EEE in accordance with the previously developed structure of criteria (Fig. 2). The developed diagnostic toolkit for evaluating the effectiveness of mathematical training within the electronic educational environment of the higher educational institution allows monitoring the progress of mathematical training of students of higher educational institutions in the context of e-learning. efficient established diagnostics of the progress of mathematical training within the framework of the developed model is focused on improving the educational process by identifying gaps in the mathematical training of students and further correcting the learning process, planning the next stages of teaching mathematical subjects, motivating students by encouraging them to succeed in mastering mathematical material.

Conclusion
Thus, achieving the effectiveness and the quality of education is the most important task of any educational institution. One of the main directions of their improvement is e-learning focused on students' active cognitive efforts in the electronic educational environment. The effectiveness of mathematical training within the electronic educational environment depends on the compliance with the regulatory and organizational, psychological and pedagogical, software and hardware, communication, and personnel requirements for the arranging of training. At the same time, the evaluation of effectiveness should meet such universal principles as specificity (clear defining the criteria-based assessment, approaches to its 2592Vera I. Toktarova, Svetlana N. Fedorova, Olga A. Kalimullina, Zarina C. Takuova / Proceedings IFTE-2020 measurement), integrity (ensuring the full volume of requirements for the learning outcomes), and producibility (soundness of methodological and technological tools for obtaining evaluation information and performing the necessary calculations). In general, it can be said that an optimally designed system of evaluating the effectiveness of mathematical training of students in the conditions of EEE will undoubtedly ensure the effectiveness of the educational activities of the higher educational institution, facilitate the access to the European educational space and increase the competitiveness of graduates in the labour market.