Special Section: E-Learning World-Learning management systems bring integration, analytics to enterprise

By Judith Lamont

Learning management systems (LMSs), software solutions that track and manage training, are beginning to fulfill their potential for bringing an enterprise perspective to training. Increased use of e-learning, integration with other enterprise systems and the introduction of analytics now offer the promise of achieving a long-sought goal, that of measuring the impact of training on business performance.

The first learning management systems were developed primarily to automate record-keeping in instructor-led training, replacing inefficient manual systems. The LMS kept track of who was registered, whether the student had completed the course and course grades. On that basis alone, the cost of the LMS was justified, because it brought order and efficiency to the management of class work. Later, the LMS was used to track students’ progress in competencies. In addition, the LMS can be used to trigger the delivery of e-learning content. An immediate benefit of that capability is the enterprisewide reach that the LMS infrastructure provides.

Gundersen Lutheran is a healthcare network that includes hospitals, group medical practices, regional community clinics and other services. It has significant training needs related to OSHA requirements, IT training and medical education. Each department throughout the healthcare network is responsible for delivering its own training, and each was keeping manual records for online and classroom training. Because Gundersen did not have a centralized training department, it was not possible to keep track of training activities in the organization as a whole. After exploring various options, Gundersen chose the Pathlore LMS as a cost-effective solution that had the functionality the organization needed.

Once the LMS infrastructure was in place, Gundersen began acquiring and distributing courseware. The first learning content the hospital acquired was a set of OSHA courses from Vivid Learning Systems.

“The courses arrived on CDs and we followed the wizard to load them on the server,” says Jay Fernandez, HRIS manager at Gundersen. “Within 45 minutes we had 14 courses available throughout the organization.” The hospital also uses courses from SmartForce (purchased by SkillSoft, for its Microsoft product training. Custom content is developed by an instructional designer at Gundersen. Templates are available to subject matter experts (SMEs) to transfer their expertise into courses.

Fernandez particularly appreciates the system’s ability to deliver and track compliance with federal regulations such as confidentiality as directed by the Health Insurance Portability and Accountability Act (HIPAA). Once a year, each employee is sent a notice with a link to a policy document, which the employee reads and signs electronically.

“As people see what the system is capable of,” notes Fernandez, “demands for additional applications have grown.” Gundersen is now in a position to better leverage its content, minimize licensing fees and ensure compliance across its many departments.

“It’s very helpful for companies to be able to get an overall view of their skills and competencies,” says Steve Thomas, CEO of Pathlore. “They can then see whether they are able to execute a proposed strategic plan, or whether they have gaps. In the event they are missing required skills, training can be put in place."

Global Campus

After an organization has discovered the value of leveraging the LMS infrastructure across the enterprise to gain a broader view of training activities and staff competencies, a valuable step is to integrate the LMS with other enterprise systems. The first integration is frequently with a human resources information system (HRIS). That allows an individual’s training history to be a part of his or her other personnel records.

APL Ltd (APL). is a global shipping and logistics company with a history that dates back to the Gold Rush. For approximately the last 30 years, the company has focused on containerized shipping and is a leader in the intermodal shipping industry. Among its many innovations are a redesign of railroad cars that allows them to carry two containers stacked vertically; the first Web site in the container-shipping industry and numerous IT solutions that enable customers to manage their shipments and supply chains. Now owned by Neptune Orient Lines (NOL), the NOL Global Campus (the corporate university) is the central NOL Group initiative for employee development.

In April 2002, the Global Campus deployed the Docent learning management system (LMS) to more than 10,000 employees worldwide. “We integrated the Docent LMS with PeopleSoft’s human resource database,” says Donna Nelson, manager of the learning management system at APL, “so that data from all employees automatically populated the LMS.” That allowed the employees to immediately begin logging on, using identification and authentication information from PeopleSoft. Past transcript information was also imported, including historical information from instructor-led training.

The Global Campus offers a variety of courses, including instructor-led, internally developed online courses, and third-party vendor online courseware on topics from “Refrigerated Cargo” to “Optimizing E-mails at Work.” In addition, online training is provided to sales staff so they can assist customers in using the APL Web site to access information about the status of shipments. The LMS automates the process of delivering required courses. Employees can view their educational records, and managers can view the courses their staff members have completed.

Kinko's rollout

Integration with an HR system also allows organizations to map training against learning tracks. Kinko’s, a leading provider of document and business services, urgently wanted to move its training “into the 21st century.” The company had a set of training programs that was inconsistent across its widely dispersed facilities and an outdated LMS. After an extensive search, Kinko’s selected the Saba Enterprise Learning Suite as the backbone for its new training initiative, the Kinko’s Knowledge Network. That rollout was one of the company’s largest and also one of its smoothest, with full deployment in just ten weeks.

The Knowledge Network learning environment includes a mix of online and instructor-led training, virtual classrooms and targeted job aids. The delivery mode is determined after interviews with the content subject matter experts, who provide information on the goals and objectives of the training. A solution with the highest probability for success is then selected, using its Delivery Media Decision Tool (DMD) which takes into account content stability and complexity, frequency of utilization, dissemination speed, audience and location as well as other metrics. More than 2,000 courses are available, including training in sales, leadership, technology and operations.

Approximately 20,000 users have ordered nearly 100,000 courses since the system’s introduction in late 2002, and each employee accesses the system an average of eight times per week.

“We are now identifying position-specific learning tracks and will ultimately map content to competencies,” says Whitney Shelley, director of learning and development at Kinko’s. “As each company project is launched, the training components will be linked to business-specific goals in order for us to accurately measure the return on investment.”

Integration with PeopleSoft is an important component of the Knowledge Network, Shelley points out, because the system immediately knows each individual’s job position and associated learning track. When new employees first log on, they are immediately identified by the LMS and receive the appropriate learning track assignments such as “general team member + manager + specific position.” Gap analyses will compare required competencies to current capabilities and track completion information. In addition, the company will integrate information about competencies with business performance measures, to provide feedback about training effectiveness.

That ability, to measure the impact of training on business performance, is probably the greatest motivator for integrating training information with other enterprise data. Although very few organizations are at the stage of actually correlating training data with operational data, many that have implemented learning management systems are beginning to think about it.

Duke's solution

Duke University Health System (DUHS) is a non-profit, fully integrated academic healthcare system dedicated to providing patient care, professional education and biomedical research. The system includes three hospitals, ambulatory surgery centers, primary and specialty care clinics and other services such as home care and wellness centers. It is integrated with the Duke University School of Medicine and the Duke University School of Nursing. Terry Seelinger, e-learning manager with Corporate Information Services at DUHS, has developed an e-learning service that has centralized control of technology but empowers client groups to focus on content development and delivery.

The system uses Learning Space from IBM Lotus as the backbone and structure for coursework, and will be implementing the new IBM Lotus LMS in the near future.

“We can do some reporting but want a greater ability to analyze the data,” says Seelinger. Once the LMS is in place, Seelinger expects to have a more robust system because they will be tapping into the LDAP enterprise directory. “This will enable us to easily analyze groups of employees to see how they are doing on their training, or query users in certain budget categories.”

It is important to distinguish analysis from the reporting functions that may be included in an LMS. Many types of predefined reports are available from LMS data or can be generated by reporting tools. However, they are static rather than being user-driven, and they do not provide the ability to correlate one measure with another. Analytical tools, whether applied just to training data or to a combination of training data and operational data, let users manipulate the data and discover relationships between different measures.

The IBM Lotus Learning Management System, announced in January 2003, is an open standards platform built on J2EE, and can be used with DB2, Oracle or SQL. It ships with the IBM directory server LDAP, which means that the system will pick up all the directory information about each staff person, including authorizations.

“Each individual can go to his or her personalized home page,” says Margaret Driscoll, market development manager for IBM Lotus, “and find out what courses are recommended, or receive e-mails about timing and availability of courses.” Course notifications can automatically be added to the calendars of Lotus Notes or Microsoft Exchange users, and threaded discussions can be added to course materials.

The LMS links with other enterprise systems through application programming interfaces (APIs) and Web services. “If a company has a CRM system that keeps track of call handling time, we can correlate performance data with training data to look for a relationship between training and on-the-job performance," Driscoll says.

In addition, through the Discovery Server, users can locate experts in different fields. The LMS also includes a basic authoring tool designed for subject matter experts, who can specify metadata tags by which the training module can be located, or can embed key words to launch a dynamic search of the Discovery Server to locate other information related to the course topic.

PeopleSoft, whose HRIS is often the one with which training information is integrated, introduced its own learning management product. “We think that learning fits into every business process in an organization, and that it needs to be triggered on a real-time basis,” says Jason Averbook, director of global product marketing for PeopleSoft’s Human Capital Management Division. Its new Enterprise Learning Management (ELM) is similar to an LMS, but is seamlessly integrated with the company’s other enterprise solutions.

“PeopleSoft has always been able to store training results,” says Averbook, “but we also wanted to be able to deploy training and, in addition, relate it to operational measures.” The products with which the ELM is integrated include its Human Resources Management System (HRMS) and CRM, as well as financial and supplier relationship applications. The PeopleSoft HRMS Warehouse is the environment in which information is collected for ad hoc reporting and complex analyses from those various sources.

Maturity model

THINQ Learning Solutions, Inc. has been active in promoting the concept of integrating with other enterprise systems, citing the benefits of being able to plan and manage learning as a “strategic initiative with a focus on performance outcomes.” The company has also developed a Learning Management Maturity Model (LM3). The LM3 describes the evolution of the learning environment from an ad hoc stage in which an organization has no consistent process for managing learning, to an optimized work force in a performance-centered organization. In between, the organization achieves “integrated performance,” in which employees understand how they contribute to the organization’s success, and learning is tightly coupled with work activities.

Until training effectiveness is measured with respect to organizational performance, it will not be meaningful, according to Michele Cunningham, VP of marketing and product management at THINQ. “Considering how training has often been evaluated in the past, you might as well report on how many pounds of employees you trained,” she contends.

Cunningham believes that to some degree, the mindset of the HR community is a limiting factor. “They are less familiar with the language of business cases and ROI,” she says.

But several THINQ customers are beginning to make correlations between their training programs and business performance. CVS/pharmacy, for example, is finding that by offering training that leads to improved career path opportunities in the company, employee retention rates are increasing. In addition, online training for the national pharmacy technician certification exam has increased employee pass rates for the exam. Both of these changes have a direct impact on the bottom line.

Mission planning

LMS products with open architectures allow users to export data to business intelligence tools for detailed analyses. In the Air Force, the Air Education Training Command (AETC) has developed the Technical Training Management System (TTMS), using an LMS from Plateau Systems. TTMS manages and keeps track of individuals in different training environments, including technical training for aircraft systems maintenance, intelligence training and a variety of basic skills for personnel. Many different skills are involved in doing those jobs. Plateau’s LMS provides the ability to organize and summarize that data, which allows the training staff to manage resources and time.

Data is also exported from TTMS into a decision support system called the AETC Decision Support System (ADSS) for further analysis. ADSS gathers data from TTMS and puts it in data cubes that can be analyzed using Cognos, a business intelligence tool.

“Since the Plateau system is integrated with Cognos,” says Milton Turner, C4I systems manager, “we can provide multiple data perspective views to find out how our training activities compare to expected planned staffing needs.” For example, if a commander in the field asks for an individual with a particular skill, AETC planners can see how many trained individuals will be available at a given point in time. This allows for mission planning, prediction of staff shortages and other resource management options.

In addition, the integration with Cognos allows easy response to ad hoc requests, such as developing statistics on job categories or demographic groups.

“We put the data into ‘Cognos Cubes,’ which allow multiple views of the same data depending on our needs,” says Turner. “In a matter of minutes we can do things that would have taken us a week if we had to write SQL queries.” “Our approach is to make our product as open as possible,” says Plateau’s Chairman and CEO, Paul Sparta, “so it can integrate easily with leading enterprise applications.” Plateau has a strong relationship with Documentum and interoperability with business intelligence tools such as Cognos, MicroStrategy, and Business Objects .

“Using the J2EE architecture allows these other applications to interoperate with every aspect of Plateau’s product,” adds Sparta. For example, if an individual’s job position changes in the HR system, that is immediately reflected in the LMS.

The ability to analyze

Within the past year, several vendors have incorporated analytics into their LMS solutions. These products, too, provide the ability to correlate data and perform ad hoc queries. Saba Analytics, part of the Saba product suite, was introduced in April 2003 and offers enterprise reporting, analysis and information distribution of data in the LMS. It includes a variety of business analyses, including cost analyses for training, content utilization and effectiveness, and the ability to view training information by business unit, customer, location and job type. It is designed for business users and provides for point-and-click charting, an analyses dashboard and e-mail subscriptions to metrics.

Being able to analyze enterprise learning activities is important from many perspectives.

“Many large organizations spend the bulk of their training dollars acquiring content,” says Malcolm Hobbs, VP of Worldwide Marketing at Saba, “yet they may not have a way to see if or how the content is being used.”

In the same way that the early learning management systems were cost-effective by automating administrative processes, this next step can help organizations use their training resources more effectively.

At a more sophisticated level, analytics will help managers evaluate the relationship between training and performance.

“Over time, the real value of training will be based on its impact on organizational objectives such as the achievement of sales quotas or product line revenue,” says Hobbs. “We believe that demonstrating this relationship will be core to the human capital development and management market, and will ensure that people throughout the enterprise are receiving the right training at the right time."

Docent Analytics was introduced in October 2002. Docent’s approach was to develop modules for specific functions, such as sales performance analysis and channel effectiveness analysis. The modules incorporate operational information in addition to LMS data to determine the impact of training activities.

“The driver for these new products was demand from our own customers,” says Tobin Gilman, VP of product and field marketing at Docent. “They were becoming increasingly interested in demonstrating that training projects would achieve defined business objectives.”

Having the ability to isolate the impact of specific training activities on business metrics elevates learning to “a more strategic, and mission-critical business process,” maintains Gilman.

GreenPoint Mortgage, which has been a Docent customer for five years, is in the process of deploying Docent Analytics to measure the impact of training on sales productivity. One key performance measure that GreenPoint selected was loan processing error rates. By using Docent Analytics, GreenPoint managers will be able to compare performance of those who have completed training modules and those who have not. The analytics can drill down to the question level on tests to see which ones correlate to performance indicators. Once training information is integrated with other enterprise data in a way that allows analytics, it will be possible to evaluate the impact of training on organizational performance.

“Training is the last major organizational function to go through a re-engineering process and move to the enterprise level,” observes Brandon Hall, president of brandon-hall.com. Interestingly, software itself may provide the basis for linking training and business performance. “Structures and processes usually do not exist within the organization to formally tie training initiatives back to strategic objectives,” says Hall. “E-learning solutions that provide a systematic method of making such a connection may help achieve this long-awaited goal.”

LMS supports Montana Department of Justice

The Montana Department of Justice manages the Criminal Justice Information Network (CJIN) and other information systems used by state law enforcement agencies. The Criminal Justice Information Services (CJIS) Bureau’s Training, Audit and Policy Section monitors access to the CJIN and provides training to more than 1,800 users. To improve availability of training and speed of delivery, the Department sought an e-learning solution. In 2001, DOJ implemented the Pathlore Learning Management System.

One of the key training needs was for certification in using the information systems. Since the CJIN system connects to the FBI’s National Criminal Justice System, users of Montana’s system must get certified within six months and recertified every two years. The FBI also audits usage to make sure proper procedures are being used when employees access the information systems. Keeping up with changing requirements can be a challenge. For example, the FBI notified the department that every user of CJIN had to read a particular letter issued to all states by a federal judge on system misuse. In the past, that requirement would have entailed mailing the document and manually keeping track of the users that have read the letter. With Pathlore, the bureau put the information in a short online course and built in an electronic acknowledgement.

When new versions of information system software are deployed, training is also required. “In 1999, when we implemented a new system, we were on the road for three months,” says Jennifer Viets, CJIS training specialist. “We went from city to city, demonstrating the software and training two people per agency.” Still, not everyone had heard about the new system. With the availability of online training, everyone can have access to instruction.

“Our last major upgrade, which took place in 2002, was the easiest we’ve ever had,” adds Viets. Training is developed by three in-house designers, who use templates to make the courseware consistent. At present, the department is using Crystal Reports to analyze usage, including the number of courses completed each month, learner scores and certification information. In the future, the department hopes to do more detailed breakdowns such as identifying what questions are being missed, so that training can be revised or refocused as necessary.

Integration within a KM framework

By including multiple applications within its mGen Enterprise solution, mGen took a somewhat different tack in integrating e-learning with other functions. Its integrated suite of products includes a learning content management system (LCMS), human capital management system (HCMS), news and information, and knowledge communities. Users can access their courses from a dynamically generated home page that reflects their level of rights and privileges. Through the HCMS, managers can query organizational skills and certifications. The news and information solution allows authoring and distribution of internal information and receipt of external news feeds that are tailored to each user’s information needs. The knowledge communities module provides threaded discussions, workflow and document management, and identification of subject matter experts.

The U.S. Department of Agriculture (USDA) is using mGen Enterprise to train personnel in the Animal Plan Health Inspection Services Agency (APHIS) how to respond to issues associated with rare, infectious animal diseases that threaten the U.S. food supply. The need for this initiative was driven in part by 9/11, which prompted APHIS to set up a new Incident Command Center. Part of the effort includes identifying resources available in the cities, towns, counties and states located near an incident. mGen Enterprise will provide a centralized database that includes those resources, and a blended learning environment of online courses and classroom certifications. APHIS is in the process of developing 29 courses for the system.

Other organizations at USDA have used mGen Enterprise for several years. For example, since 1999, the USDA’s Office of Communications has used mGen’s organizational communications and readiness system to deploy and track congressionally mandated training for U.S. dairy farmers and brokers. Other USDA agencies such as Rural Development now use the existing infrastructure provided by mGen Enterprise to deliver training. The ability to leverage an existing enterprise e-learning infrastructure across multiple departments is a big advantage of making the initial investment, and improves overall ROI.

mGen has its roots in interactive multimedia distribution systems, and has a strong capability in digital asset management.

“We set out in 1997 to include an LMS as part of larger system that encompassed knowledge management and organizational communications,” says Jack Battersby, president of mGen. “Also, we wanted to support both structured learning and the more informal learning that is an important part of organizational knowledge.” mGen Enterprise includes an authoring tool designed for subject matter experts, but is also compliant with standards (SCORM and AICC) and can distribute other content.

Judith Lamont is a research analyst with Zentek Corp., e-mail jlamont@sprintmail.com.