Quality Metrics in Corporate Training

There are many performance metrics in corporate training and quality, and each training program requires an individual approach. However, there are common challenges, pitfalls, and principles worth considering. Corporate training experts shared their cases and experiences at the first online meetup of the Digital Learning community and Yandex.Q, titled "Education." We have prepared a summary of their presentations to help improve educational processes in companies.

What determines training quality

Mikhail Sverdlov, Development Director at Skypro and curator of the "Education" community on the Yandex.Q platform, noted that the lack of measurement of product quality metrics and their connection to financial results leads to inefficiency. He compared this to a log floating downstream or timber being rafted down a river, emphasizing the importance of a conscious approach to product management. Measuring key performance indicators helps avoid a situation where a company stagnates and cannot develop effectively.

Metrics play a key role in assessing the quality of training. To understand what exactly can be considered "quality training," it is necessary to consider several key criteria. The discussion participants highlighted the following aspects:

the training outcome meets business expectations;
employees consciously choose training and see its benefit—that is, they perceive it not as an obligation, but as a useful tool for professional and personal development;
training directly leads to the planned result for all stakeholders, including participants and the business customer;
training allows the business to accelerate—optimize processes, quickly and effectively "integrate" employees into the necessary processes;
the process and outcome of this training are of interest to at least someone.

In this text, we will consider the experiences of the participants who shared their specific approaches. Participants discussed various strategies that have proven effective in their practice. These approaches can be useful for those seeking new solutions and methods in their field. We will analyze key ideas and recommendations that will help improve results and achieve goals.

How to evaluate a course before it starts

Learning type: asynchronous, e-learning.

Asynchronous learning is a flexible approach to learning that allows students to study materials at their own pace. E-learning courses in this format offer access to a variety of resources, including video tutorials, interactive assignments, and quizzes. This learning method promotes deeper learning, as students can review material as needed and at their own pace. Asynchronous e-learning courses are ideal for busy individuals who want to combine study with work or other commitments. Choosing asynchronous learning opens up opportunities for self-education and skill development in various areas.

At this stage, preparation for the upcoming activities takes place. It is important to carefully analyze all necessary materials and resources to ensure the successful completion of the project. The preparatory stage includes gathering information, defining goals and objectives, and developing a strategy. This process will help avoid potential mistakes and misunderstandings in the future, which in turn will improve work efficiency. Pay attention to every aspect of preparation to lay a solid foundation for achievement.

The quality assessment of an e-course should begin at the development stage, before it is presented to employees. This opinion was expressed by Ulyana Ravedovskaya, Director of Training at VTB Digital Academy. Early attention to the quality of course content and structure allows us to identify shortcomings and improve the training material, which ultimately contributes to more effective training and employee development.

The assessment of training quality is often based on the final results. When training is completed and we see the desired result, a sense of success and high quality arises. However, with e-learning courses, such evaluation comes at a late stage. The effort invested, significant financial outlays, and the participation of numerous people do not guarantee success. If we discover that a course does not deliver the expected results, this leads to significant losses. Therefore, it is important to develop methods for assessing learning effectiveness in advance, allowing for early detection of problems. Predictive metrics help reduce costs by enabling the timely identification and correction of errors and deficiencies. They provide the opportunity to replace ineffective exercises and edit texts, which contributes to improving the overall quality of the content and increasing its effectiveness. The use of predictive metrics is becoming a key tool for optimizing processes and achieving better results.

Photo: Nikita Sursin / Shutterstock

The expert shared her experience of using assessment in the Corporate Rosatom Academy, where she previously headed the methodology department. A similar system is planned to be implemented at VTB Academy in the near future. This approach will improve the quality of educational programs and enhance the effectiveness of training, taking into account the modern needs of the corporate sector.

Each course was evaluated by an expert and methodological council, which included the Academy's management, including the CEO. Initially, the council consisted of ten members, which, according to the speaker, contributed to the development of a unified position on the quality criteria of educational products. Over time, course quality assessment was transferred to a three-person commission, whose composition was also regularly updated, but it remained exclusively composed of management. This approach ensured high standards and consistency in the evaluation of educational programs.

Courses were evaluated using various criteria to determine their quality and effectiveness. Important aspects of the evaluation included course content, instructor qualifications, teaching methods used, accessibility of materials, and student feedback. These factors help potential students make an informed choice and select the most suitable courses for their needs. Systematic course evaluation helps improve the educational process and increase student satisfaction.

Specific and realistic course goals and objectives.

Objectives should be consistent with the expected changes in knowledge, skills, and attitudes. While clarity of formulation is usually not an issue, the realism of objectives is often questionable. Often, even short-term courses are set ambitious goals, such as developing three complex skills, which is unrealistic. It is important to carefully consider the topic of goals and objectives, as this can significantly improve the effectiveness of training, the expert emphasized.

The benefits of interactive exercises.

Training should be aimed at enabling employees to analyze and understand the information received during the course, as well as receive detailed feedback. It is important that practical exercises aimed at developing knowledge and skills are indispensable, believes Ulyana Ravedovskaya. These exercises should be clearly linked to the course's goals and objectives, which will improve the effectiveness of the training and ensure its relevance to modern requirements.

According to the expert, a common mistake is to use formal exercises in a course that can be completed by simply clicking buttons, without taking the course itself. This is an ineffective approach. What is really necessary? Ulyana clarifies: "If we are striving for a specific result, the set of exercises should contribute to achieving this result."

Relevance and objectivity of the course content.

The course content should be consistent with the goals and objectives, and reflect the current state of science and the opinion of the author of the tools or approaches. Particular attention should be paid to the latter aspect. Often, developers do not verify the content, uploading their interpretation of an approach or tool without checking the original sources. This can lead to misinformation among course participants. It is important to ensure accurate and reliable information so that participants gain relevant knowledge and skills.

Concise formatting, simple language, and the absence of contradictions are important.

The course should avoid long sentences that are difficult to understand on first reading. Avoid using complex abbreviations or separating lists into separate paragraphs. This will help make the material more accessible and understandable for participants.

Photo: Sergii Gnatiuk / Shutterstock

The transition to a new learning format is a serious challenge, especially for those accustomed to synchronous learning. In a traditional approach, we quickly respond to participants' misunderstandings by restating the material, explaining it in new terms, or answering any questions that arise. However, in a new course, this may not be possible. Therefore, it is essential to make the material as clear as possible the first time, eliminating any inconsistencies. One common example is the use of different terms for the same concept in different parts of the course, which can be confusing. It is important that all terminology is consistent and unambiguous to avoid confusion and ensure effective learning. The course should include examples and real-world cases to enhance understanding. Practical examples help reinforce theoretical knowledge and demonstrate how to apply it in real-world situations. It is important that course participants have the opportunity to see how theoretical concepts are implemented in practice, which facilitates deeper learning and increases its value. Realistic case studies will help participants develop their own strategies and approaches, which will contribute to their professional growth and the successful application of acquired knowledge in the future.

Clarity of course graphic design.

All icons and buttons should be standardized, which ensures consistency and improves the user experience. Important elements should be highlighted to attract attention and increase their visibility. It is recommended to use no more than three different fonts in the design, which helps to create a harmonious and professional design. Standardizing interface elements, highlighting key components, and limiting the number of fonts are important aspects that contribute to the creation of an effective and user-friendly website.

Ease of navigation - in three clicks.

The learner should be able to quickly return to exercises and the necessary information. Creating training accessible in three clicks is a real art, as the speaker noted. This is critical, because the longer an employee searches for the necessary materials, the less likely they are to successfully assimilate knowledge and increase their confidence. An effective organization of the learning process promotes deeper understanding and successful self-assessment.

Ulyana Ravedovskaya emphasized that predictive metrics demonstrate a high correlation with employee feedback and NPS levels. These metrics are also comparable to the content of comments left by students when evaluating the academy as a whole. This confirms the importance of using predictive metrics to analyze satisfaction and the quality of the educational process.

Designing corporate training programs is a key aspect of employee development and improving business performance. It is important to consider the needs of the organization, as well as the individual requests of employees. To successfully create a training program, it is necessary to analyze the current skills of employees and identify areas that require improvement.

The first step in designing training programs is identifying business needs. This can be done through surveys, interviews, or performance analysis. Based on the information collected, a program structure should be developed that will include both theoretical and practical modules.

The next step is choosing training formats. This can include in-person, distance learning, or blended formats. The use of modern technologies, such as online platforms and interactive tools, helps increase employee engagement in the learning process.

It is also important to implement a program effectiveness evaluation system. Regular monitoring and feedback will help adapt training to changing conditions and business requirements.

Therefore, designing corporate training programs requires a comprehensive approach that considers the needs of the company and employees, as well as current educational trends. This not only ensures the improvement of professional skills but also contributes to the creation of a more effective and motivated team.

Why and How to Measure Engagement

Type of training: synchronous online. This format allows students and teachers to interact in real time, which promotes deeper assimilation of materials and active participation in the learning process. Synchronous online training provides the opportunity to ask questions, participate in discussions, and receive feedback immediately, which increases the effectiveness of training. This approach is ideal for both individual lessons and group courses, providing flexibility and accessibility of education.

The midterm stage is an important moment in the process of implementing a project or achieving a goal. At this stage, an analysis of the current state is carried out, strengths and weaknesses are identified, and further steps to achieve the final result are determined. The midterm stage allows you to adjust the strategy, if necessary, and make changes based on the collected data. This is a time to evaluate progress, discuss results, and plan next steps, which contributes to the successful completion of the project. Effective management during the midterm phase helps mitigate risks and optimize resources, ultimately resulting in a higher-quality outcome.

Engagement is one of the key metrics in synchronous learning. However, it is important to evaluate not only the actions and results of students, but also those of teachers and trainers. Modern technologies significantly simplify this process, especially online. Tamara Kocharova, CEO of Training Space and Development Director of Oblako Group, shared this case study. Her experience demonstrates how the effective use of technology can increase engagement and improve learning outcomes.

In 2020, she and her colleagues initiated a long-term training program for 200 executives at a large manufacturing company. Ten trainers were responsible for implementing the training, working according to a uniform standard and scenario. However, the interim results after the first month of training were lower than expected by both the program developers and the client. This prompted a review of our teaching approaches and methods, which allowed us to further improve the program's effectiveness and achieve our goals.

The platform used for training allowed us to conduct a detailed analysis of student behavior. We were able to track the frequency of switching to other browser tabs, as well as the completion rate of training sessions. The results showed that program participants leave the online room approximately 20 minutes before the end of the session. This data allows us to better understand student engagement and optimize the learning process to increase its effectiveness.

Photo: Nikita Sursin / Shutterstock

During the analysis of the trainers' work, their ability to follow a single scenario, as well as the level of participant engagement and learning outcomes. It was found that the duration of sessions varied: one trainer conducted a lesson for almost two hours, while another limited it to an hour and a half. Some trainers omitted key elements of the training scenario. These facts formed the basis for further analysis and hypotheses. Based on the training sessions, feedback questionnaires allowed us to identify both leaders and laggards among the trainers. Three out of ten specialists did not meet the company's established quality standards.

An interesting aspect was revealed: the level of student engagement was 60%, while the average satisfaction rating from the feedback questionnaires reached 8.8 out of 10. This indicates that students highly value the quality of training, but are not fully immersed in the process. This gap between satisfaction and engagement may indicate the need to improve teaching methods and create a more interactive educational environment.

Following the evaluation, the project team initiated supervision and a retraining program for digital trainers, and introduced session timing controls. These measures had a positive impact on engagement rates, which increased by 23%. However, participant satisfaction increased only slightly—by 0.8%. Tamara Kocharova noted that if the training quality assessment had been based solely on participant feedback, one could conclude that no changes had occurred and that problems never existed. Therefore, relying on feedback alone is insufficient for a comprehensive assessment of training quality. It is important to consider comprehensive data and conduct regular analysis to ensure sustainable development and improvement of training programs.

The quality of the results obtained does not meet expectations. We focused on this issue and realized that relying solely on questionnaires to evaluate synchronous sessions within trainings is not enough. It is important to use deeper metrics than simple questions about whether participants liked the event or whether they will attend the next training. Modern platform solutions allow for more accurate measurement, significantly improving the evaluation process and helping to tailor training to the needs of participants.

The engagement of training participants has a direct impact on the final results achieved by the client. One of the key factors determining engagement is the qualifications of the trainers. Tamara recommends carefully evaluating the performance of online trainers. Regardless of the platform you use—Zoom, Teams, or Training Space—evaluation is essential, as it affects engagement levels. Namely, engagement directly affects the quality of training.

Reworked text:

Please pay attention to the additional materials:

In the modern world, online educational events are becoming increasingly popular. For their successful organization and implementation, it is necessary to use the appropriate tools. Let's look at 12 effective tools that will help you create high-quality online educational events. The first tool is Zoom, which offers high-quality video communication and a variety of features for interacting with participants. Google Meet is also popular due to its simplicity and integration with other Google services. For creating interactive presentations, you can use Prezi, which allows for dynamic elements and visual effects. If you need to organize webinars, it's worth considering the WebinarJam platform, which offers a wide range of event management features. Microsoft Teams is also an excellent choice for educational institutions, providing opportunities for collaboration and communication. Kahoot can be used to create polls and quizzes, which helps make learning more engaging. Mentimeter is also suitable for interactive audience interaction and collecting feedback in real time. Trello can be used to manage projects and tasks, which will help organize your workflow and keep track of deadlines. Google Classroom is a great tool for managing the learning process and interacting with students.

Canva will help you create visually appealing materials for your events, and Slack will provide convenient communication and information sharing between participants. Finally, you can use OBS Studio for recording and editing videos, which will significantly improve the quality of your online events.

Using these 12 tools will allow you to organize high-quality online educational events, making them more effective and interesting for participants.

How to assess quality using Bloom's taxonomy

Blended learning is an effective approach to the educational process, combining traditional methods with modern digital technologies. This format allows students to gain knowledge both in the classroom and through online platforms. Blended learning promotes deeper assimilation of the material, as it gives students the opportunity to study at their own pace. The integration of various learning formats helps develop independent information retrieval and critical thinking skills. In today's educational environment, blended learning is becoming increasingly popular, providing flexibility and accessibility to knowledge for students of various levels.

Stages: midterm and final. The midterm stage includes key checks and adjustments that help assess progress and make necessary changes to the project. The final stage represents the completion of the work, where the results are summarized, the results are analyzed, and the final report is compiled. Both stages are essential for achieving a successful outcome and ensuring the quality of the project.

One of the cases examined concerned mandatory blended learning. This was discussed by Maria Ovchinnikova, Head of Management Competency Development at Burger King.

To assess the quality of mandatory programs required for employee transitions to new positions, the company's specialists developed metrics based on Bloom's Taxonomy. Assessment is carried out at several levels, allowing for a more in-depth analysis of material and skill acquisition. This approach ensures a systematic assessment of employees' knowledge and skills, which in turn helps improve their professional competence and readiness for new challenges at work.

Knowledge.

Learning effectiveness is determined through surveys and tests integrated into the learning process. This means that knowledge is assessed not only at the end of the course, but also during its implementation. Trainers conduct measurements to identify moments when it is necessary to return to the material being studied and help trainees assimilate key concepts and information. Maria Ovchinnikova explains that this approach ensures a deeper understanding and retention of the material.

Understanding.

Understanding of the material is assessed through testing, which is conducted following the training activities. Moreover, the passing score is set not at the beginning, but during the training, which allows for a more accurate assessment of the level of knowledge acquisition.

We have assumed possible passing score values. Our employees began taking tests, and based on an assessment of the difficulty and discrimination of the questions, we determined the actual level. As a result, we set a target of 80% successful completion of tests for both courses and training sessions," the expert commented.

Manifestation and Analysis.

At this level, a behavioral indicator metric was implemented, developed by training specialists in collaboration with the company's internal experts. Assessment is carried out using checklists completed by managers, analyzing their employees' behavior over specific periods after completing training. For this purpose, specific behavioral indicators were identified, which are assessed on a scale of "manifested," "manifested irregularly," "manifested consistently," and "manifested under certain conditions." This system allows for effective tracking of employee progress and identification of areas requiring additional attention.

Maria noted the importance of tracking the number of behavioral indicators that consistently manifest themselves after training, based on measurement results.

There are specific metrics that go beyond the traditional taxonomy, one of which is timeliness. The quality of employee training is determined not only by its content but also by the timeframe and timeframe required for preparation, as set by the business. Timeliness plays a key role in the effectiveness of training, as delays can negatively impact employee performance and adaptation.

How to Assess the Quality of Employee-Choice Training

Training Type: Optional. Mandatory training often requires strict adherence to a schedule and content, while optional training gives participants the opportunity to choose what and when to study. This approach allows people to tailor the educational process to their individual needs and interests. Optional training can include online courses, seminars, webinars, and other formats that promote self-education and skill development. This is an excellent way for professionals to expand their knowledge and competencies without the pressure of strict deadlines and obligations.

Evaluation stages can be divided into midterm and final. Midterm stages allow you to track progress and make adjustments during the project. Final stages summarize all the work and provide an analysis of the results achieved. These stages are important for understanding the effectiveness of completed tasks and for further planning. Midterm assessments help identify problems early, which contributes to a higher-quality project completion. Final stages, in turn, help evaluate overall success and identify areas for future improvement. Properly separating intermediate and final stages promotes a more structured approach to project management and increases the chances of successfully achieving goals. The metrics for mandatory training are clear, but the question arises of how to evaluate the quality of programs that employees choose at their own discretion. Evgeniya Rashchupkina, head of the Skillbox School of Methodologists and founder of the LXD hero project, shares her opinion on this matter. She emphasizes the importance of analyzing and evaluating such programs to improve the effectiveness of employee training and development. The School of Methodologists is an initiative of the Skillbox Methodology Department, aimed at creating a space for self-development for specialists in the field of methodology. Training at this school is not mandatory, but rather a set of useful synchronous activities. These events include workshops, support clubs, a Buddy Club, and meetings for sharing experiences. These formats help methodologists develop their skills, expand their professional horizons, and strengthen connections in the community.

Evgenia explained that methodologists use various approaches and tools to assess the quality of education at schools. The primary focus is on analyzing learning outcomes and monitoring the educational process. Methodologists conduct regular inspections and research to identify the strengths and weaknesses of the educational process. This allows them not only to assess the quality of education but also to develop recommendations for its improvement. Using modern methods and technologies in quality assessment helps create a more effective educational environment.

Target metrics.

This category includes:

CSAT, that is, how satisfied participants are with the educational activities;
the number of authors and speakers. For example, in the fourth quarter of 2021, they focused on encouraging methodologists to speak more at professional events and write expert articles. That is, not only consuming expert content, but also creating it by sharing experiences.

Leading metrics.

Training coverage includes the total number of employees participating in the process, as well as attendance at synchronous events, which shows how many invited people actually attended. In addition, an important indicator is the average number of events per participant. These parameters help evaluate the effectiveness of the educational program and employee engagement in the training.

The speaker presented an interesting metric that can indirectly indicate the quality of the training product. This metric is primarily focused on product development paths, helping to determine which elements should be prioritized and which can be abandoned. Using such a metric can significantly improve training effectiveness and make the product more attractive to users, contributing to its further improvement.

Evgenia decided to conduct an analysis of the product's compliance with market demand, known as product-market fit. To do this, she administered a survey to the participants of the School of Methodologists, asking how they would feel about changing the number of opportunities to develop various school features. The survey results were processed and visualized using a Kano model calculator. The results were unexpected: the methodologists expressed a desire to increase the number of activities. However, Evgeniya Rashchupkina did not rely entirely on the survey results and noted that this serves as an additional reason for further analysis and reflection on the needs of the participants.

Reading is an important aspect of personal development and education. It helps improve analytical and critical thinking skills. By immersing yourself in books, articles, and other materials, you enrich your vocabulary and broaden your horizons. Reading also helps develop creativity and imagination, allowing you to see the world from different perspectives. Whether you strive for self-development or want to improve your skills, regular reading will become an indispensable tool on your path. Incorporate reading into your daily practice, and you will notice positive changes in your life.

Corporate Training: 7 Key Insights into Employee Needs

Employees are increasingly recognizing the importance of corporate training for their professional growth. However, for training to be effective, their real needs must be taken into account. In this article, we will consider seven key insights into what employees really want from corporate training programs.

First, employees seek personalized training. They want programs tailored to their specific skills and goals. The second important aspect is practical focus. Employees prefer training that includes real-world cases and practical examples, which helps them apply the knowledge they gain in their daily work.

The third insight is employees' desire to receive feedback. They want to understand their strengths and weaknesses, as well as receive recommendations for further development. The fourth aspect is access to a variety of training formats. Webinars, online courses, master classes, and coaching—employees value the ability to choose the format that best suits them.

The fifth insight concerns flexibility in training. Employees want to be able to learn at their own time and pace. The sixth aspect is the importance of management engagement. Employees feel motivated when their support and participation in training are demonstrated in practice.

Finally, the seventh insight is the desire for career growth. Employees want to see how corporate training can impact their career prospects and opportunities for advancement. By taking these insights into account, companies can develop more effective and engaging corporate training programs that meet employee expectations and foster professional development.

How Not to Measure Quality

At first glance, ready-made case studies may seem like the perfect solution, but implementation is not so simple. These examples serve rather as inspiration for developing a unique metrics system. Maria Ovchinnikova emphasized that one should not blindly adapt other people's metrics and calculation methods. It is important to formulate your own metrics based on individual goals and conditions, coordinating them with the client. It is also critical to familiarize the training team with the calculation algorithms for the selected metrics. This will improve the efficiency and accuracy of work, and ensure compliance with project expectations and requirements.