UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Development of an index of quality for the planning of management training programs Rusnell, Albert Dale 1974

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
831-UBC_1974_A2 R88.pdf [ 9.24MB ]
Metadata
JSON: 831-1.0055930.json
JSON-LD: 831-1.0055930-ld.json
RDF/XML (Pretty): 831-1.0055930-rdf.xml
RDF/JSON: 831-1.0055930-rdf.json
Turtle: 831-1.0055930-turtle.txt
N-Triples: 831-1.0055930-rdf-ntriples.txt
Original Record: 831-1.0055930-source.json
Full Text
831-1.0055930-fulltext.txt
Citation
831-1.0055930.ris

Full Text

DEVELOPMENT OF AN INDEX OF QUALITY FOR THE PLANNING MANAGEMENT TRAINING PROGRAMS by ALBERT DALE RUSNELL B.A. University of Alberta, 1964 B.Ed. University of Alberta, 1966 M.A. University of B r i t i s h Columbia, 1970 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION in the Department of Adult Education We accept t h i s thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA September, 1974 RIGHTS OF PUBLICATIONS AND LOANS In presenting t h i s thesis i n p a r t i a l f u l f i l m e n t of the requirements for an advanced degree at the University of B r i t i s h Columbia, I agree that the Library s h a l l make i t fr e e l y available for reference and study. I further agree that permission for extensive copying of t h i s thesis for scholarly purposes may be granted by the Head of my Department or by his representative. I t i s understood that copying or publication of t h i s thesis for f i n a n c i a l gain s h a l l not be allowed without my written permission. ABSTRACT The purpose of t h i s study was to develop an Index of Program Planning which might r e l i a b l y and v a l i d l y measure the standard of qu a l i t y present among a set of program planning procedures o r d i n a r i l y exhibited by a company i n the development of i t s managerial and supervisory t r a i n i n g programs. As the f i r s t phase, a conceptual model of program planning for t r a i n i n g i n industry was developed, and i t consisted of seven major planning functions: (1) corporate analysis and manpower planning (2) determining and assessing t r a i n i n g needs (3) designing the tr a i n i n g program (4) l e g i t i m i z a t i o n (5) i n s t r u c t i o n a l support (6) evaluation (7) maintenance of behaviour. Eight statements were then constructed for each of the seven functions to specify behaviours which might be performed by a company i n the conduct of that function. A panel of sixty-nine judges participated i n three structured exercises which were designed to test those f i f t y - s i x statements. The Index of Program Planning consisted of twenty-six statements from the o r i g i n a l f i f t y - s i x which met the c r i t e r i a for each of the three exercises. The c r i t e r i a were as follows: (1) the statement was a v a l i d example of the planning function i t represented (2) the major planning function of the statement was generally agreed upon by a panel of judges (3) the importance of the statement i n the planning process was generally agreed upon by a panel of judges. A numerical value which represented an o v e r a l l standard of quality for program planning a c t i v i t i e s was calculated, using the index, by combining the weighted value of importance and i i and the r e l a t i v e per cent of use by a company for each of the twenty-six s p e c i f i e d planning behaviours. The highest score possible on the index was 505.3, and scores were greater i n di r e c t proportion to a combination of the greater number of d i f f e r e n t planning behaviours generally used, the greater importance of those behaviours, and the more extensive use of those behaviours over a period of time. A panel of one hundred large corporations i n Vancouver, B r i t i s h Columbia, was selected for the purpose of estimating the r e l i a b i l i t y and v a l i d i t y of the index. Twenty-three companies conducted th e i r own l o c a l management t r a i n i n g programs, and were interviewed for the purpose of testing the Index of Program Planning. The r e l i a b i l i t y of the index was estimated using a one-way analysis of variance procedure, and the c o e f f i c i e n t of r e l i a b i l i t y was found to be +0.85. The construct v a l i d i t y for the index was estimated through the use of three hypotheses which included twenty-six independent variables. Better qu a l i t y program planning was s i g n i f i c a n t l y associated with a greater per cent of tr a i n i n g conducted during company time, a greater per cent of managers and supervisors p a r t i c i p a t i n g i n the programs, the number of days allocated by a company for managers to attend programs, the greater willingness of a company to spend money for tr a i n i n g programs, the more favorable attitude of top management toward training, a lesser per cent of company reimbursement for employees enrolled i n external educational courses, a greater per cent of t r a i n i n g s t a f f who had received some formal t r a i n i n g i n an educational f i e l d , and a greater per cent of tr a i n i n g s t a f f with more than two years experience i n corporate t r a i n i n g . The construct v a l i d i t y iv for the index was j u s t i f i e d on the l o g i c a l consistency which was evident, indicating that better quality program planning was found among companies which placed more emphasis and commitment on their own i n t e r n a l management tra i n i n g programs. TABLE OF CONTENTS CHAPTER PAGE I. INTRODUCTION 1 The Problem 1 Purpose of the Study 5 D e f i n i t i o n of Terms 6 Concepts related to tr a i n i n g and development.. 6 Concepts related to program planning . . . . 7 Concepts related to variables i n the study . . 8 Design of the Study 9 Development of the Index of Program Planning . 9 Survey of Companies 11 Scope of the Study 15 Plan of the Report 16 II . DEVELOPMENT OF THE PROGRAM PLANNING MODEL . . . . 17 Estimating Excellence i n Program Planning . . . 18 The Model 22 Major Program Planning Functions 2 9 Corporate Analysis and Manpower Planning . . 30 Determining and Assessing Training Needs . . 31 Designing the Training Program 32 Legitimization 33 Instructional Support 34 Evaluation 35 Maintenance of Behaviour 36 II I . DEVELOPMENT OF THE INDEX OF PROGRAM PLANNING ... 38 Judgements of Statements 39 Selection of Statements 48 The Index of Program Planning 52 v i CHAPTER PAGE IV. ANALYSIS AND VALIDATION OF THE INDEX OF PROGRAM PLANNING 66 Tests of Developmental Procedures 66 Method of successive i n t e r v a l s 66 R e l i a b i l i t y of opinions 67 Survey of Companies 69 The panel 71 R e l i a b i l i t y and V a l i d i t y 73 R e l i a b i l i t y 73 V a l i d i t y 75 Hypothesis one 76 Hypothesis two 77 Hypothesis three 80 Interpretation of the hypotheses 82 V. SUMMARY AND CONCLUSIONS 85 Summary 86 Conclusions 94 BIBLIOGRAPHY 98 APPENDIX A 103 APPENDIX B 12 3 APPENDIX C 12 8 APPENDIX D 144 APPENDIX E 150 v i i LIST OF TABLES TABLE PAGE I. Program Planning A c t i v i t i e s Considered Important by Various Authors 26 II . Conceptual Schema of Functions for Program Planning . 28 I I I . Results for V a l i d i t y and Categorization Exercises 41 IV. Values of Q and Median Values of Importance for Exercise Three 53 V. The Index of Program Planning 61 VI. Relationships of Functions and Statements i n the Index of Program Planning . . . . . 65 VII. Disposition of Interview Requests for Companies in Survey Panel 71 VIII. Nature of Business for Companies Conducting Management Training 72 IX. C o e f f i c i e n t of Internal Consistency for the Index of Program Planning 74 X. S t a t i s t i c a l Analysis for Hypothesis One 77 XI. S t a t i s t i c a l Analysis for Hypothesis Two 79 XII. S t a t i s t i c a l Analysis for Hypothesis Three . . . . 82 v i i i ACKNOWLEDGEMENTS I would l i k e to express my appreciation to the members of the committee for th e i r continuing assistance, advice, and support over the period of thi s study. I would also l i k e to acknowledge the p a r t i c i p a t i o n and intere s t i n the study on the part of those individuals who acted as judges, and on the part of those corporations which volunteered their assistance. Without help from those individuals and companies t h i s study would not have been possible. In addition, I would like" to' thank the Extension Division, University of Saskatchewan, for the provision of time to conduct interviews with companies i n Vancouver. CHAPTER ONE INTRODUCTION Personnel development and t r a i n i n g have become important functions within business and industry. Most corporations are concerned with preventing the knowledge and s k i l l s of their people from becoming obsolescent, and with developing new knowledge, attitudes, and s k i l l s among employees so that the organization may prosper i n the modern environment of technological change. Each year large numbers of personnel engage i n company-sponsored educational a c t i v i t i e s which are conducted under such t i t l e s as Management Development, Supervisory Training, or Job Training. It has been estimated that up to 35 b i l l i o n d o l l a r s i s spent each year by corporations i n North America for the development and t r a i n i n g of t h e i r personnel. (19) In some instances companies administer personnel development budgets which are larger than those of their l o c a l municipal school systems. Naturally, each firm i s concerned that i t i s obtaining the best value for i t s expenditures of both time and money, and any information which might lead to more e f f e c t i v e procedures and programs for human resource development i s sought for i t s p o t e n t i a l value. THE PROBLEM Because of the increasing costs of employee training,? and the growing r e a l i z a t i o n of i t s necessity for corporate survival, management groups have been demanding information with respect to the procedures and benefits associated with company involvement i n human resource development. In response to the concerns of 1 2 management, concentrated e f f o r t s have been launched to f i n d or develop more adequate personnel development procedures, but the search for improvement has been hampered by a lack of tested knowledge. The f i e l d of t r a i n i n g i n industry i s r e l a t i v e l y new and very l i t t l e research has been conducted with respect to i t . The small body of knowledge which has been developed i n r e l a t i o n to the personnel development function i s primarily the r e s u l t of i n d i v i d u a l t r a i n i n g directors describing t h e i r a c t i v i t i e s or making generalizations on the basis of past experience. A series of guidelines r e l a t i n g to successful t r a i n i n g practices has grown from the interchange of ideas between personnel development s p e c i a l i s t s , but that information has not been systematically tested nor widely disseminated among the business community. A summary of those guidelines was provided by Cone, who recommended twenty-four practices among the following categories: (1) plan and set objectives (2) determine costs (3) develop content (4) select trainees (5) determine approach and method (6) make s t a f f i n g decisions (7) evaluate t r a i n i n g (8) keep records and reports. (18) Despite the existence of such guidelines for program planners, however, t r a i n i n g programs i n industry are often not as e f f e c t i v e as they might be. The effectiveness of t r a i n i n g within companies has been documented i n the occasional research report, such as one by Hannon, who interviewed executives from one hundred and f o r t y - s i x United States i n d u s t r i a l companies and reported the following ratings of the work done by their corporate t r a i n i n g departments: Excellent - nine per cent, Good - t h i r t y -s i x per cent, Average - twenty-six per cent, Only Fair - seventeen 3 per cent, Poor - one per cent. (29) Other indications of tr a i n i n g d i f f i c u l t i e s are evident from descriptions of programs that have f a i l e d , an example of which was provided by Kozoll, who described and analyzed the problems of a supermarket tr a i n i n g program that was eventually discontinued. (35) Generalizations from the experiences of t r a i n i n g s p e c i a l i s t s have also provided insight into the lack of effectiveness of some t r a i n i n g programs. Turpin stated that most management tr a i n i n g programs that f a i l e d could be traced to the following factors: (1) content was not applicable to participants' jobs (2) program was contrary to company p o l i c i e s and practices (3) applicants r e s i s t e d and took a negative approach to the t r a i n i n g program. (54) Some of the more common reasons for curriculum or program f a i l u r e were l i s t e d by L e s l i e This as including: (1) the trainees do not need the t r a i n i n g (2) the trainees need the t r a i n i n g but do not know i t or w i l l not admit i t (3) the problem leading to the establishment of a s p e c i f i c t r a i n i n g program i s not one that can be solved by ex i s t i n g t r a i n i n g knowledge and/or techniques (4) the trainer i s not knowledgeable or doesn't present h i s material properly (5) the material being presented i s too d i f f i c u l t to transfer and apply, or i s too abstract to be r e a d i l y applicable to the job s i t u a t i o n of the trainee (6) neither the trainee's p a r t i c i p a t i o n i n the program nor his "changed job behaviour" i s adequately reinforced by h i s superiors, associates or work s i t u a t i o n (7) the trainees need the training, know they need i t , but r e s i s t i t for other reasons. (50) Other indications of the lack of success by tr a i n i n g operations i n business and industry are available, but as i s generally the case when a complete t r a i n i n g department i s eliminated from an organizational structure, d e t a i l s are r a r e l y reported i n the l i t e r a t u r e . 4 Of the many problems present within personnel development a c t i v i t i e s i n business and industry, few are as obvious as unsuccessful or i n e f f e c t i v e t r a i n i n g programs, and of the many functions performed by personnel development authorities, few are as evident as the planning and implementation of educational events or programs. The importance of the planning process i n r e l a t i o n to t r a i n i n g programs has been recognized by such authors as DePhillips, Berliner, and Cribbin, who suggested that, " i n order to accomplish s a t i s f a c t o r i l y anything of importance there must be planning i n advance of doing", (24:19) and by Campbell, who i n the f i r s t annual review of Personnel Training and Development for the Annual Review of Psychology noted that there are several types of papers which re-appear i n the l i t e r a t u r e at regular in t e r v a l s , and that one of those types argues that t r a i n i n g should be well-planned and systematic. (14:566) Despite the v i s i b i l i t y and importance of the program development r e s p o n s i b i l i t y , however, objective research with respect to the program planning process i s scarce. Although there are a large number of t r a i n i n g textbooks and a r t i c l e s which suggest that good planning i s necessary, or which provide model methods for developing programs, or which re l a t e case h i s t o r i e s of program planning, very l i t t l e research or systematic testing of the accumulated guidelines has been reported. For example, there appears to have been l i t t l e or no research conducted to compare the state of program planning practices among a number of d i f f e r e n t corporations, nor have a number of t r a i n i n g s p e c i a l i s t s been surveyed with respect to th e i r opinions of which s p e c i f i c planning procedures may be more ef f e c t i v e than others, nor has there been any research r e l a t i n g 5 to conditions within organizations which appear to be associated with better program planning procedures for t r a i n i n g programs. Most important, however, from the viewpoint of the present study has been the lack of an instrument which might objectively measure the standards of q u a l i t y for program planning procedures used by a number of d i f f e r e n t companies. Many business and i n d u s t r i a l concerns are attempting to improve th e i r personnel development practices at the present time, and more sophisticated corporate t r a i n i n g a c t i v i t i e s are envisioned for the future, but those.objectives w i l l l i k e l y require more extensive research i n both the p r a c t i c a l and th e o r e t i c a l areas of tr a i n i n g and personnel development. Because the problems of i n e f f e c t i v e t r a i n i n g programs are highly v i s i b l e within companies, and since the program planning process may offer some assistance to tr a i n i n g s p e c i a l i s t s i n the search for better programs through an elimination of d i f f i c u l t i e s commonly associated with i n e f f e c t i v e programs, research within the program planning area i s relevant to the corporate search for better tr a i n i n g practices and procedures. I t i s with respect to that area of concern the present study was focussed. PURPOSE OF THE STUDY This research project explored the processes and procedures used i n business and industry for the planning and implementation of management and supervisory t r a i n i n g programs. The major purpose of the study was to develop an instrument which might be used to measure the standard of qual i t y among a set of procedures o r d i n a r i l y used by a corporation i n the development of formal tr a i n i n g programs for managerial and supervisory employees. The task demanded an accumulation and integration of 6 the ex i s t i n g guidelines for the development of such programs, the establishment of objective c r i t e r i a against which the qu a l i t y of corporate planning procedures might be measured, and the estimation of r e l i a b i l i t y and v a l i d i t y measures with respect to the instrument which was developed. The importance of the study was evident, considering the p o t e n t i a l uses of such an objective instrument for measurement i n t r a i n i n g research of both p r a c t i c a l and t h e o r e t i c a l nature. With a c a p a b i l i t y for objectively measuring the standards of q u a l i t y with respect to program planning processes among d i f f e r e n t corporations, some of the areas presently lacking i n research might be explored within the f i e l d of personnel development. For example, the state of program planning practices might be compared among a number of corporations for purposes of establishing norms of planning quality, the association between standards of planning q u a l i t y and other organizational factors might be tested for purposes of building theory i n personnel development practice, and the relationships between program effectiveness and q u a l i t y of planning procedures might be explored for purposes of program improvement. DEFINITION OF TERMS Several key terms were defined to d i f f e r e n t i a t e between the general case and the more s p e c i f i c instances where phenomena were c l o s e l y associated with each other. The terms defined were related to t r a i n i n g and development, program planning, and the s p e c i f i c variables included i n t h i s study. Concepts Related to Training and Development Personnel Development - any d i r e c t action s p e c i f i c a l l y designed and implemented by an organization to change the 7 behaviour of any of i t s members by imparting information or providing for the practice of s k i l l s . The term may be used synonymously with "human resource development". Training - actions taken by an organization to supply s p e c i f i c knowledge, s k i l l s , or attitudes which are required to meet given standards of job performance. It i s concerned with a s p e c i f i c task or subject area as a r e s u l t of some deficiency or inadequacy i n the performance of a job. Concepts Related to Program Planning Program - a series of t r a i n i n g a c t i v i t i e s or meetings within a formal i n s t r u c t i o n a l setting. Generally, each meeting i s r e l a t i v e l y short and the topic of concern i s limited i n scope. At least three meetings or six hours of i n s t r u c t i o n are required to constitute a program. The word "course" may be used as a synonym for "program". Program Planning - the a c t i v i t i e s which are performed to enable a t r a i n i n g program to be implemented and completed, except for those a c t i v i t i e s which involve the d i r e c t i n t e r a c t i o n between the i n s t r u c t i o n a l agent and the learner p a r t i c i p a n t within the formal i n s t r u c t i o n a l setting. Another term which may be used synonymously i s "program development". Program Planning Function - one of a group of related actions which make up the program planning process. Functions are the major categories into which s p e c i f i c behaviours are c l a s s i f i e d . Examples of program planning functions include the assessment of t r a i n i n g needs, design of the program, and evaluation, each of which may be performed i n a number of d i f f e r e n t ways. Program Planning Task - s p e c i f i c behaviours which may be performed as d i s t i n c t elements of functions i n the planning 8 process. For example, job analysis and manpower inventory are tasks which may be performed as part of the function of assessing t r a i n i n g needs. Concepts Related to Variables i n the Study Quality of Program Planning - the standard of excellence for a set of corporate program planning behaviours as measured by a set of objective c r i t e r i a . Index of Program Planning - an instrument which might be used to measure the qua l i t y of program planning within a corporation i n the development of i t s managerial and supervisory t r a i n i n g programs. Higher degrees of excellence i n program planning are indicated by higher scores on the Index of Program Planning. The basic objective of t h i s research study was to develop the Index of Program Planning. Economic Power and Influence - the a b i l i t y of a corporation to p o s i t i v e l y or negatively a f f e c t the economic well-being of a community or area along with i t s resident population as a consequence of corporate decisions and subsequent actions i n the conduct of company a f f a i r s . Commitment to Training - a favorable attitude of the organization and i t s management toward training, as indicated by a willingness to consider t r a i n i n g an important i n t e g r a l part of normal business operations. Degree of Development of Company Training Systems - the degree to which the company t r a i n i n g system has established i t s e l f as a f u l l y functioning aspect of normal corporate operations by means of the extensiveness of i t s f a c i l i t i e s , expertise of i t s personnel, and provision of i t s services to the organization. 9 DESIGN OF THE STUDY The study was conducted i n two major phases. Phase One included the development of the Index of Program Planning, an instrument to measure the q u a l i t y of program planning i n corporate t r a i n i n g . The second phase of the study consisted of a survey of companies within the greater Vancouver area of B r i t i s h Columbia, for the purpose of estimating the r e l i a b i l i t y and v a l i d i t y of the Index of Program Planning. Development of the Index of Program Planning The most important elements of the study were related to the f i r s t phase, where the Index of Program Planning was created. The developmental steps commenced with the conceptualization of a model o u t l i n i n g seven major functions of the program planning' process for t r a i n i n g programs i n business and industry. Those seven functions were as follows: (1) corporate analysis and manpower planning (2) determining and assessing t r a i n i n g needs (3) designing the t r a i n i n g program (4) l e g i t i m i z a t i o n (5) i n s t r u c t i o n a l support (6) evaluation (7) maintenance of behaviour. Following the development of the program planning model and the d e f i n i t i o n of i t s constituent elements, the t r a i n i n g l i t e r a t u r e was searched for i l l u s t r a t i o n s of corporate t r a i n i n g behaviours which were considered to be examples of the seven separate functions included i n the model. For each function, eight statements were constructed to specify behaviours which might be performed by a company i n the conduct of the function which was being, i l l u s t r a t e d . Two separate exercises were then devised to test the f i f t y - s i x statements which had been constructed. A number of 10 people who were knowledgeable with respect to corporate t r a i n i n g matters or to program planning theory were then located across western Canada and were asked to p a r t i c i p a t e i n the study as a panel of judges. In the f i r s t exercise, a group of judges was asked to consider the v a l i d i t y of each statement with respect to the function i t was intended to represent, and i n the second exercise, another group of judges was asked to c l a s s i f y the f i f t y - s i x statements according to appropriate function. C r i t e r i a were a r b i t r a r i l y established for the two exercises, and the objective of those exercises was to eliminate statements which were not considered to be v a l i d examples of the functions they were to represent, or statements for which there was no agreement with respect to the i r general purpose i n the planning process. A l l statements from the o r i g i n a l l i s t of f i f t y - s i x which met the c r i t e r i a for both of the o r i g i n a l two exercises were included i n a t h i r d exercise. In the t h i r d exercise, a large group of judges was asked to estimate the importance of each stated behaviour i n the program planning process. The objective of the t h i r d exercise was to select a number of statements for the Index of Program Planning on the basis of the degree of agreement among judges with respect to the importance of the stated behaviours i n the development of management and supervisory t r a i n i n g programs within companies. Where the agreement among judges was not s u f f i c i e n t , statements were eliminated from further consideration. The Index of Program Planning was constructed from a l l statements which were not eliminated i n any of the three exercises. Each of the statements included i n the instrument, as a r e s u l t of testing procedures i n the three exercises, met 11 the following c r i t e r i a : (1) the statement i s a v a l i d example of the function i t represents (2) the function of the statement i n the program planning process i s generally agreed upon by a panel of knowledgeable judges (3) the importance of the statement i n the program planning process i s generally agreed upon by a panel of knowledgeable judges. Furthermore, each of the statements selected for the Index of Program Planning had attached to i t a value of importance, which was the median score of importance i n the planning process, as suggested by the panel of judges. In i t s completed form, the Index of Program Planning consisted of a number of stated behaviours, each with an associated value of importance i n the planning process. By combining the value of importance and the proportional use of each selected behaviour i n the program planning sequence, an o v e r a l l score which represented the standard of quality for a set of corporate planning a c t i v i t i e s was calculated with the Index of Program Planning. Survey of Companies The second phase i n the study was designed to estimate the r e l i a b i l i t y and v a l i d i t y of the Index of Program Planning which was developed i n the f i r s t phase. To undertake that task, a survey of companies was conducted i n which the Index of Program Planning was used repeatedly and i n which a var i e t y of data were c o l l e c t e d from each p a r t i c i p a t i n g corporation. For purposes of the survey a panel of one hundred companies, each of which was thought to employ two hundred people or more i n the greater Vancouver area was selected from the 1968 Canada Manpower Centre Area P r o f i l e  for Metropolitan Vancouver (10) and the 1971 B r i t i s h Columbia  Trade Directory. (15) The personnel manager, t r a i n i n g director, 12 or another appropriate person within each organization was approached, and interviews were conducted with those who agreed to p a r t i c i p a t e i n the survey. Only those corporations which conducted their own managerial or supervisory t r a i n i n g programs i n B r i t i s h Columbia were able to respond to the Index of Program Planning and the data from those companies were used to estimate the r e l i a b i l i t y and v a l i d i t y of the measurement instrument. Companies which did not conduct t r a i n i n g programs i n B r i t i s h Columbia were unable to provide appropriate information for purposes of the survey, and only abbreviated interviews were conducted i n those instances. The interview schedule which was used for the survey included, along with the Index of Program Planning, a series of questions related to the company i n general and to i t s t r a i n i n g system i n B r i t i s h Columbia. An estimate of r e l i a b i l i t y was obtained for the Index of Program Planning through an analysis of corporate responses to i n d i v i d u a l items within the index, as well as the t o t a l scores which were calculated from those i n d i v i d u a l responses. An analysis of variance procedure, as outlined by Winer, was used to derive the estimated c o e f f i c i e n t of r e l i a b i l i t y for the Index of Program Planning. (57:124) Ordinarily, the construct v a l i d i t y for the Index of Program Planning as a measure of qu a l i t y for program planning procedures within managerial and supervisory t r a i n i n g might have been estimated through c o r r e l a t i o n of the r e s u l t s from the Index of Program Planning with other e x i s t i n g measures or known observables representing degrees of q u a l i t y i n program planning, but because no such measures had been previously established, the estimation of construct v a l i d i t y for the Index of Program Planning was attempted through a series of hypotheses. Three hypotheses were constructed, each of which was considered to be associated with q u a l i t y of program planning for t r a i n i n g i n business and industry. With respect to format, each hypothesis had as i t s major dependent variable the quality of program planning, as represented by corporate scores derived from the Index of Program Planning, and an association was postulated between that variable and a conceptual independent variable for which several d i f f e r e n t measures were possible. The association between the qu a l i t y of program planning and the va r i e t y of d i f f e r e n t measures within each hypothesis was then tested. Depending upon the nature of the data, either regression or analysis of variance techniques were used i n the testing procedures, and the .10 l e v e l of s t a t i s t i c a l significance was chosen as a c r i t e r i o n l e v e l . Hypothesis One: Better quality program planning w i l l be found among corporations with greater economic power and influence. The f i r s t hypothesis was an attempt to determine the r e l a t i o n s h i p between the quality of program planning and the economic power and influence of corporations, as might be indicated by the number of employees, location of the head o f f i c e , amount of the company pay r o l l , and value of sales for products or services. I t was f e l t that larger corporations would l i k e l y exhibit better planning through the provision of resources and personnel which were more adequate for the task than might have been so among less powerful companies. Hypothesis Two: Better qu a l i t y program planning w i l l be found among corporations which exhibit greater commitment toward the t r a i n i n g and personnel development function. The concern of the second hypothesis was the r e l a t i o n s h i p 14 between the qua l i t y of program planning and the commitment toward personnel development and tra i n i n g functions exhibited by corporations. Measures of corporate commitment included the existence of written t r a i n i n g p o l i c i e s , amount of company time allowed managers and supervisors for t r a i n i n g purposes, amount of expenditure allowed for personnel development a c t i v i t i e s , attitudes of top management toward company tr a i n i n g programs, p o s i t i o n within the organization of the person responsible for tr a i n i n g matters, number of years i n which managerial and supervisory t r a i n i n g programs were continuously provided within the company, and the degree of management and supervisory p a r t i c i p a t i o n i n tra i n i n g a c t i v i t i e s . I t was f e l t that better quality planning would be associated with greater commitment toward t r a i n i n g as a consequence of previous successes i n personnel development a c t i v i t i e s which may have been responsible for achieving the commitment of management toward training, and also as a consequence of the support and expectations of a qua l i t y product which might accompany the implied importance of t r a i n i n g programs where commitment i s evident. Hypothesis Three; Better q u a l i t y program planning w i l l be found among corporations with more highly developed t r a i n i n g and personnel development systems. The t h i r d hypothesis dealt with the existence of a s p e c i f i c t r a i n i n g organization within the company, and with i t s success i n establishing i t s e l f as a stable and important element i n the organization. An association was postulated between the quali t y of program planning and the degree of development within the t r a i n i n g system, as measured by the number of tra i n i n g s p e c i a l i s t s employed by the company, educational expertise 15 within the s t a f f , degree of t r a i n i n g experience among the s t a f f , number of programs offered, and amount and a l l o c a t i o n of the t r a i n i n g budget. I t was f e l t that better q u a l i t y planning would r e s u l t from a more highly developed t r a i n i n g system which might involve more sophisticated procedures and more competent s t a f f members i n the tr a i n i n g a c t i v i t i e s . While no s p e c i f i c tests were devised for other forms of v a l i d i t y , as were the three hypotheses devised to estimate construct v a l i d i t y , aspects of both content and face v a l i d i t y were considered to be adequately s a t i s f i e d as a r e s u l t of the procedures r e l a t i n g to the model development, selection of statements, and panel judgements which were undertaken throughout the development of the Index of Program Planning. SCOPE OF THE STUDY This research project attempted to develop an Index of Program Planning which might be considered v a l i d within a western Canadian setting. The majority of t r a i n i n g s p e c i a l i s t s who acted as judges for the project were residents of B r i t i s h Columbia, Alberta, or Saskatchewan, and the responses made by those people for the developmental exercises may have been conditioned by their experiences r e l a t i n g to t r a i n i n g matters i n western Canada. A similar set of exercises s i m i l a r l y conducted i n another region might have provided an e n t i r e l y d i f f e r e n t set of values of importance for the statements, or an e n t i r e l y d i f f e r e n t set of statements for the Index of Program Planning. With respect to the survey of companies, the scope of the study was evident, as the panel of companies was drawn e n t i r e l y from the greater Vancouver area of B r i t i s h Columbia. In addition, the scope of the study was limited to companies employing between two hundred people and f i f t y thousand people. 16 PLAN OF THE REPORT The report began with an overview of the research study, including i t s purposes, procedures, and scope. The second chapter provides a review of the l i t e r a t u r e which was used to develop the program planning model and the series of statements for sections of that model. A det a i l e d explanation of the development of the Index of Program Planning constitutes the t h i r d chapter, while the re s u l t s from the survey of companies are included i n chapter four. A f i f t h chapter concludes the report with a summary and suggestions for further research which may be required as a r e s u l t of the study. Materials used i n the conduct of the project are included i n the appendices. CHAPTER TWO DEVELOPMENT OF THE PROGRAM PLANNING MODEL Program planning i n adult education encompasses a broad spectrum of concerns. Those concerns of program planning as a f i e l d of study include, i n addition to the individuals or groups representing the c l i e n t organizations and the program planning bodies, the programming procedures and processes which are used by people and their organizations to develop the programs and events which are desired. The agencies which plan educational programs for adults include u n i v e r s i t i e s , voluntary organizations, labour unions, business groups, l i b r a r i e s , public night schools, r e l i g i o u s i n s t i t u t i o n s , and government organizations, and among those diverse inte r e s t groups, programs are planned at three l e v e l s . Community programming focuses generally upon the educational needs of t o t a l communities, i n s t i t u t i o n a l programming focuses upon the goals of an i n s t i t u t i o n i n of f e r i n g a set of a c t i v i t i e s for i t s c l i e n t s , and a c t i v i t y programming focuses upon the implementation of s p e c i f i c short-term educational events by the agencies. Organizations planning programs at any of the community, i n s t i t u t i o n , or a c t i v i t y levels r e l a t e to the participant learners through i n s t r u c t i o n a l methods which may focus on the in d i v i d u a l learner, small or large groups of learners, or on the t o t a l community. In addition to the scope of program leve l s and of in s t r u c t i o n a l methods, a range of program planning processes are u t i l i z e d by the programming agencies. Those processes include the sequences of a c t i v i t i e s which are undertaken by the program planners i n designing, implementing, and evaluating their programs. 17 18 Despite a d i v e r s i t y of program objectives, s p e c i a l i s t terminologies, c l i e n t expectations, and organizational systems among the various programming agencies, a general consensus exists among adult educators with respect to a sequence of functions which are e f f e c t i v e i n the development of educational events. Six planning functions which are common to many programming agencies are as follows: (1) determine the needs of c l i e n t s (2) e n l i s t the p a r t i c i p a t i o n of c l i e n t s i n planning (3) formulate program objectives (4) design an i n s t r u c t i o n a l sequence of events (5) implement the program (6) evaluate the program. Each programming agency, while considering the common set of planning a c t i v i t i e s , o r d i n a r i l y adapts the sequence of a c t i v i t i e s to i t s own purposes. Thus, some agencies might emphasize c e r t a i n planning phases, while other agencies may complement the basic planning sequence with additional functions. For example, a g r i c u l t u r a l extension agencies may concentrate on involving t h e i r c l i e n t e l e group i n planning a c t i v i t i e s , while u n i v e r s i t i e s may focus more on additional planning a c t i v i t i e s related to selection of subject matter, and public night schools may place p r i o r i t i e s upon i d e n t i f i c a t i o n of c l i e n t e l e i n t e r e s t s . The adaptation of planning procedures to the requirements and c h a r a c t e r i s t i c s of i n d i v i d u a l agencies may be noted throughout the program planning l i t e r a t u r e , and i t i s with respect to such an adaptive process that the model for program planning i n business and industry was formulated for purposes of t h i s research project. ESTIMATING EXCELLENCE IN PROGRAM PLANNING Although numerous agencies plan programs for adults, and many of those agencies have developed sequences of planning 19 a c t i v i t i e s to meet the i r own p a r t i c u l a r requirements, few have reported any attempts to estimate the standard of excellence associated with their programming procedures. In the instances where an assessment of planning procedures was reported, either a set of subjective guidelines was established or an i d e a l planning approach was outlined as a c r i t e r i o n for excellence, and one or more programs were analyzed i n r e l a t i o n to those standards. The uses of such c r i t e r i a i n program planning were discussed by Beal, who suggested that the c r i t e r i a were appropriate both as guides to planning action and for evaluation both during and after the planning process. (4) Beal also suggested that c r i t e r i a were generally used, i f at a l l , only after programs were completed, and were therefore not being used to the f u l l e s t advantage. Within the adult education l i t e r a t u r e a majority of studies reporting estimates of qu a l i t y for program planning procedures have appeared i n the inte r e s t area of a g r i c u l t u r a l extension. With respect to educational events, Boyle suggested a l i s t of eleven planning p r i n c i p l e s with p a r t i c u l a r relevance to a g r i c u l t u r a l extension, and they were as follows: (1) o v e r - a l l objectives of the agency should be considered (2) educational needs of the p o t e n t i a l program participants should be considered (3) interests of the entire community should be considered (4) a wide range of resources should be given consideration (5) the planning group should include l o c a l c i t i z e n s who are p o t e n t i a l participants i n the program (6) democratic processes should be used wherever possible i n planning the program (7) various methods which might be used i n reaching the objectives should be explored i n the planning (8) the program planning 20 process should be continuous (9) the program planning process should allow for f l e x i b i l i t y (10) provisions should be made for appraisal and evaluation of the program (11) the planning group should coordinate i t s planned a c t i v i t i e s with those of other adult education agencies. (8) In a study of program planning within the Iowa Cooperative Extension Service, Powers reported an approach to the estimation of excellence which elaborated on the use of planning p r i n c i p l e s . (44) In that instance, an i d e a l approach to planning was developed i n committee, s p e c i f i c operational conditions to be met were defined as standards of i d e a l performance, and rat i n g scales were developed to measure the discrepancies between the id e a l and the actual planning a c t i v i t i e s i n a series of programs. As a r e s u l t of the study, Powers concluded that 22.4 per cent of the conditions were met as defined, 50.8 per cent of the a c t i v i t i e s were i n need of improvement, and 26.5 per cent of the suggested conditions received no attention. Another approach to estimating excellence i n planning was followed by Darter, who developed ten p r i n c i p l e s for program development and tested them by comparing the planning procedures of successful and unsuccessful a g r i c u l t u r a l extension programs in an intensive analysis of case studies i n two southern states of the Unites States. (22) He concluded that successful programs resulted from greater p a r t i c i p a t i o n of l o c a l people i n many phases of planning, greater coordination with other agencies, more concern with planning as an educational process, and greater concern for development of people rather than concern only for the conduct of programs. Yet another approach to estimating excellence was undertaken by a research team from the Economic Council of Canada. (32) Although the research was conducted i n r e l a t i o n to manpower planning a c t i v i t i e s i n business and industry, the method i s applicable to appraisals of program planning i n adult education. Furthermore, the Economic Council's study was c l o s e l y related to the present project i n that i t sought to estimate excellence of corporate operations within the area of human resource development. The approach taken by researchers i n that study was to seek out and report the best practices currently i n use by companies for manpower planning a c t i v i t i e s . Using those best practices as standards of excellence, a three-point scale was developed which permitted the rating of discrepancies between the i d e a l descriptions of best practices and those practices reported by f o r t y - s i x companies which were surveyed. The study concluded that of the companies surveyed, twenty per cent conformed to the best practice i n corporate planning functions, nine per cent to that top l e v e l i n job i d e n t i f i c a t i o n , t h i r t y seven per cent i n the maintenance of a manpower inventory, seven per cent i n the matching of jobs and manpower, and eleven per cent i n the t r a i n i n g and development of management and professional personnel. As representative approaches to the estimation of excellence i n planning a c t i v i t i e s , the studies c i t e d indicated that those procedures reported i n the l i t e r a t u r e have tended to emphasize a subjective appraisal of actual planning a c t i v i t i e s i n r e l a t i o n to the c r i t e r i a , and hence, an estimation of excellence i n planning which lacked numerical precision. In addition, any forms of objective testing with respect to the v a l i d i t y of the c r i t e r i a which were selected for the appraisal procedures appeared to be lacking. As a r e s u l t of the evident need for more objective approaches to the estimation of excellence i n program planning, and i n consideration of the apparent lack of appraisals with respect to planning a c t i v i t i e s for t r a i n i n g i n business and industry, the present study was undertaken. THE MODEL With respect to the program planning l i t e r a t u r e within d i f f e r e n t i n t e r e s t areas of adult education, two approaches to the design of planning models were noted: (1) conceptual functions (2) flow charts. Those planning models based upon the f i r s t approach consisted of a l i s t of major functions which were considered to be e s s e n t i a l i n the planning process; those planning models based upon the second approach consisted of s p e c i f i c planning a c t i v i t i e s represented within a series of flow-chart lines suggesting i n d e t a i l the order of a c t i v i t i e s to be followed. Although models based upon either approach might have been transformed to the other, major differences between the approaches were evident with respect to their s p e c i f i c i t y of tasks and systematic ordering of a c t i v i t i e s . Planning models which l i s t e d conceptual functions were not s p e c i f i c i n terms of a c t i v i t i e s to be performed, and no prescribed order of events was indicated, other than a normal sequence of events as suggested by progress i n the planning process. An example of a conceptual function planning model was provided by Boone, Dolan, and Shearon, i n th e i r Conceptual Schema for Programming i n the Cooperative Extension Service. (6) That model was focused at the i n s t i t u t i o n a l programming l e v e l and l i s t e d four major planning functions: (1) the i n s t i t u t i o n and i t s renewal process (2) l i n k i n g the i n s t i t u t i o n to i t s publics through need analysis and leader involvement (3) program design and implementation (4) program evaluation and accountability Within each function, a number of constituent elements and tasks were l i s t e d as factors to be considered i n operationalizing the function. Planning models based upon the flow-chart approach were very s p e c i f i c i n terms of a c t i v i t i e s to be performed, and detailed sequences of phases and feedback loops between a c t i v i t i e s were represented by the flow-chart l i n e s . A systems approach to planning, including the customary use of diagrams, appeared to be evident i n the flow-chart planning models. An example of that approach was provided by LaForest i n a model for program planning i n adult basic education. (36) The model consisted of nine major subsystems which were considered important i n planning an adult education program: (1) develop ABE program (2) quantify operating parameters (3) i d e n t i f y program needs (4) promote program (5) apply enrollment procedures (6) plan and conduct program (7) r e c r u i t and select faculty (8) t r a i n faculty (9) conduct terminal evaluation. Through a procedure of expansion i n d e t a i l , the model was outlined i n flow-chart format, where 210 subsystems and 172 relationships between the subsystems were represented by the si g n a l paths commonly used i n the systems approach to planning. In addition to those models which were based upon the two basic approaches to the design of planning models, a number of other program planning models contained elements of each approach. For example, Houle outlined seven general components of the planning process and f i t t e d them into a simple systems approach using d i r e c t i o n a l l i n e s to indicate the sequence of events as well as the feedback component. (30) His components were as follows: (1) a possible educational a c t i v i t y i s i d e n t i f i e d (2) a decision i s made to proceed (3) objectives are i d e n t i f i e d and refined (4) a suitable format i s designed (5) the format i s f i t t e d into larger patterns of l i f e (6) the plan i s put into e f f e c t (7) the res u l t s are measured and appraised. Houle indicated a feedback component between the f i r s t and l a s t a c t i v i t y . Similarly, Knowles focused upon the andragogical process of program development and outlined the following major functions within a series of d i r e c t i o n a l flow-chart l i n e s : (1) est a b l i s h climate conducive to adult learning (2) create organizational structure for p a r t i c i p a t i v e planning (3) diagnose needs and interests (4) define general purpose (5) define program objectives (6) design a comprehensive program (7) operate program a c t i v i t i e s (8) evaluate. (34) Within each major function, more s p e c i f i c tasks to be conducted were suggested. Because the flow-chart and systems approach models tended to be more pr e s c r i p t i v e and hence, less appropriate as models to be adapted and implemented i n a variety of s p e c i f i c or unique circumstances, a decision was made for purposes of t h i s study to use the conceptual function approach for the development of a program planning model. Thus, i n out l i n i n g a number of c r i t i c a l functions for planning t r a i n i n g programs i n business and industry, a s p e c i f i c order of detailed planning a c t i v i t i e s was not of concern, nor were the relationships between the functions with respect to sequence, although a general sequence of functions was suggested for the model i n accordance with progress through the planning process. As was noted, many programming agencies i n adult education agree upon the effectiveness of a general sequence of planning functions, and each agency o r d i n a r i l y adapts the sequence to i t s own purposes. Within the l i t e r a t u r e r e l a t i n g to t r a i n i n g i n business and industry, one such adaptation of functions was often mentioned as a basis for e f f e c t i v e program planning. Although the terminology used by d i f f e r e n t authors varied with respect to the suggested planning a c t i v i t i e s , a sequence of four functions was commonly included i n the l i t e r a t u r e , often as an i n t e g r a l sequence among more extensive planning models. Table I i l l u s t r a t e s some of the planning functions which were discussed by various authors, and as may be noted, the four most-often c i t e d functions were as follows: (1) determine the training needs (2) design the program (3) conduct the i n s t r u c t i o n (4) evaluate the program. Each author l i s t e d i n Table I who was associated with any of the planning functions, even though the terminology may have d i f f e r e d from that used to l a b e l the functions for t h i s study, had either included that function i n a recommended sequence of procedures, had discussed the function and i t s constituent tasks i n d e t a i l , or had implied the importance of the function i n d i r e c t l y though perhaps not as a major planning phase. Because of the consensus among authors with respect to the importance of those four planning functions, the four phases were incorporated as components of the program planning model which was developed for t h i s study. In addition to the four planning functions which were f i r s t included i n the model, three further programming functions were incorporated into the model because of the importance of t h e i r roles with respect to planning 26 TABLE I PROGRAM PLANNING ACTIVITIES CONSIDERED IMPORTANT BY VARIOUS AUTHORS1 AUTHORS PROGRAM PLANNING FUNCTIONS COMPANY DETERMINE DESIGN GAIN CONDUCT EVALUATE MAINTAIN PLANNING NEEDS PROGRAM SUPPORT PROGRAM BEHAVIOUR Annett(1) X X X Bare (2) X X X Bumstead (12) X X X Byers(13) X X X X X Connellan(19) X X X X X X Curl (21) X X X Duce(2 5) X X X X X Eitington(2 7) X X X Finn (28) X X X X House (31) X X X X X X Lawr ie & Boringer(37) X X X X McGehee & Thayer (38) X X X McNary(39) X X X X X Markwell & Roberts (40) X X X M i l l a r (41) X X X X Nadler(42) X X X Off & Boutin(43) X X X Quinn (45) X X X Renton(46) X X X X X Rose(47) X X X X Singer(49) X X X X X Tracey (51,52) X X X X X X Turner(53) X X X X X Warren (56) X X X X X 1 X indicates that the author i n some manner indicated the importance of the function i n the program planning process t r a i n i n g programs i n business and industry, as outlined by par t i c u l a r authors. Those three functions were concerned with corporate analysis and manpower planning, gaining management commitment and l e g i t i m i z a t i o n for t r a i n i n g a c t i v i t i e s , and maintenance of behaviour following the conclusion of t r a i n i n g programs. The importance of corporate analysis and manpower planning was established by Davis i n an unpublished doctoral thesis focusing upon the integration of executive development and long range corporate planning, (2 3) the importance of le g i t i m i z a t i o n was established by House i n an explanation of the commitment approach to management development (31) and by Turner i n an analysis of managerial climate for training, (53) and the importance of maintaining behaviour was proposed by Connellan i n a discussion of maintenance techniques. (19) The model of program planning which was developed for t h i s study may be seen i n Table I I . As may be noted, i t consisted of seven major planning functions and their appropriate d e f i n i t i o n s . The functions were as follows: (1) corporate analysis and manpower planning (2) determining and assessing t r a i n i n g needs (3) designing the t r a i n i n g program (4) l e g i t i m i z a t i o n (5) i n s t r u c t i o n a l support (6) evaluation (7) maintenance of behaviour. Each function was defined i n terms of the purposes for which i t was included i n the planning process, and i n consideration of the a c t i v i t i e s and tasks associated with the function by various authors. Thus, corporate analysis and manpower planning was related to the planning of o v e r a l l corporate objectives and the estimation of manpower requirements which were necessary to meet those objectives, determining and assessing t r a i n i n g needs was related to the study of corporate 28 TABLE II CONCEPTUAL SCHEMA OF FUNCTIONS FOR PROGRAM PLANNING 1. CORPORATE ANALYSIS AND MANPOWER PLANNING A study of company objectives, plans, and structure and of the company's r e l a t i o n s h i p to changes i n society for the purpose of estimating how the company might change and what kinds of personnel might be required for the company to succeed i n the future. 2. DETERMINING AND ASSESSING TRAINING NEEDS A study of company operations and company personnel for the purpose of determining where problems exi s t at present or w i l l l i k e l y e x i s t i n the future, and an assessment of the p o t e n t i a l roles of tr a i n i n g i n helping to overcome those problems. 3. DESIGNING THE TRAINING PROGRAM Planning and designing a sequence of i n s t r u c t i o n a l a c t i v i t i e s to achieve the desired t r a i n i n g goals and then producing necessary program materials and making preparations for implementation of the program within the company. 4. LEGITIMIZATION Gaining the approval of company management for the t r a i n i n g program, including management agreement that the program i s f i n a n c i a l l y worthwhile, that the content of the program i s acceptable, and that the procedures for conducting the program are within company regulations. 5. INSTRUCTIONAL SUPPORT The provision of services and personnel to f a c i l i t a t e the i n s t r u c t i o n a l process as the t r a i n i n g program i s underway. 6. EVALUATION The assessment of the t r a i n i n g program i n terms of the effectiveness of the t r a i n i n g system which produced i t , or i n terms of the achievements of the participants, or in terms of resultant e f f e c t s of the program on the company as a whole. 7. MAINTENANCE OF BEHAVIOUR Attempting to ensure that behaviours learned i n the tr a i n i n g program are ca r r i e d over into the work s i t u a t i o n . problems and the assessment of tra i n i n g as a solution to those problems, designing the tr a i n i n g program was related to the setting of objectives and development of a program to meet those objectives, l e g i t i m i z a t i o n was related to gaining the approval of corporate management for the tra i n i n g program, i n s t r u c t i o n a l support was related to the provision of administrative and planning services for the program once underway, evaluation was related to the appraisal of the program and the personnel involved i n i t , and maintenance of behaviour was related to the continuance of newly-learned behaviours i n the work environment at the conclusion of the tr a i n i n g program. MAJOR PROGRAM PLANNING FUNCTIONS Upon completion of the program planning model, confirmation was sought that the functions chosen for the planning model were i n d i v i d u a l l y considered to be important phases within t r a i n i n g procedures. For that purpose, the tra i n i n g l i t e r a t u r e was searched for evidence of materials r e l a t i n g to each of the seven major planning functions. The amount of material found varied greatly among the functions, with a larger number of works focused on the areas of evaluation, need assessment, and program design, and lesser amounts of material focused on the remaining four areas. In most instances, however, the l i t e r a t u r e was available i n s u f f i c i e n t quantity so that only a sampling was considered feasible for purposes of t h i s study. In addition to confirmation that the seven planning functions were considered by various authors to be unique e n t i t i e s , the l i t e r a t u r e was used as a source of ideas for s p e c i f i c tasks to represent each of the functions i n further phases of the project. 30 Corporate Analysis and Manpower Planning One major corporate goal i s good economic performance, as measured by p r o f i t . The employment of high q u a l i t y personnel i s a r e q u i s i t e of p r o f i t , and the continued prosperity of an organization: often depends upon an approach to long-term planning which ensures that the appropriate numbers and types of personnel are available to the company when they are needed. Any long-term planning e f f o r t usually includes a present inventory and a future forecast of the organizational structure and i t s personnel. Those reports may be derived from the consideration of corporate p o l i c i e s , plans, and future objectives as well as from estimates of future economic and technological changes. Other factors which are important to the reports include inventories of knowledge, s k i l l s , aptitudes, and attitudes of employees, and consideration of retirements, promotions, and losses through dehiring and personal mobility. A comparison between the present manpower inventory and the estimated future manpower requirements and performance s k i l l s , according to such authors as Black (5), Coleman (17), and Vetter (55), w i l l indicate the personnel development which i s necessary for the attainment of future organizational objectives. The general outcome of the organization and manpower analysis i s o r d i n a r i l y a set of personnel development objectives which may be used to coordinate manpower development programs with organizational objectives. The corporate analysis function i s a more generalized form of the assessment of t r a i n i n g needs function i n that the tr a i n i n g and development needs of the company are ascertained from the analysis, but the corporate analysis i s a document for t o t a l company growth and i t i s concerned with far more than the 31 personnel development aspect alone- Unfortunately, the corporate analysis function i s ra r e l y integrated with the personnel development system of companies. Davis found that of 225 American companies from Fortune's 500 l i s t , almost none integrated the long-range planning and executive development functions. (2 3) Determining and Assessing Training Needs The tr a i n i n g and development needs of corporations may be c l a s s i f i e d as developmental or maintenance types, according to Barrett. (3) Developmental needs r e l a t e to the future personnel requirements of the company, as indicated by the res u l t s of the corporate analysis and long-term planning a c t i v i t i e s . Maintenance needs, on the other hand, re l a t e to present problems and operational d e f i c i e n c i e s as are indicated by unmet business objectives. When i t becomes evident that problems or de f i c i e n c i e s e x i s t i n the business, an analysis must be undertaken to determine their precise nature and location. After the problem has been defined, decisions are made with respect to the nature of the solution and the adequacy of resources to deal with the s i t u a t i o n . I f the r e s u l t s of those decisions place the r e s p o n s i b i l i t y for action within the tr a i n i n g and development sphere, and allow for the necessary funding, then an analysis of the problem i s undertaken by tra i n i n g s p e c i a l i s t s as a foundation from which to develop solutions. McGehee and Thayer indicated the fundamental importance of an objective assessment of tr a i n i n g needs by stating, "Once facts can be secured as to the actual t r a i n i n g needs of an employee or group of employees, then t r a i n i n g can be conducted e f f e c t i v e l y and economically. The research approach to determining 32 tr a i n i n g needs requires time and costs money, and t r a i n i n g also requires time and costs money. Unless research and research r e s u l t s are used i n planning and implementing t r a i n i n g the time and money spent on t r a i n i n g i s e a s i l y wasted." (38:121) The research and analysis of t r a i n i n g needs may be performed by such techniques as observation of work, review of records, formal analyses of tasks and performances, or by personal contacts, and i t may be conducted at the organizational, job, or personal l e v e l s . When the analysis i s complete the differences between the e x i s t i n g and the i d e a l s i t u a t i o n may be indicated by a set of t r a i n i n g objectives. Those objectives serve as the focus of a l l subsequent tr a i n i n g and development a c t i v i t i e s . Designing the Training Program The purpose i n designing a t r a i n i n g program i s to combine subject areas and methods of i n s t r u c t i o n which w i l l operate within the corporate environment to change the behaviour of the i n d i v i d u a l or group toward the behavioural s p e c i f i c a t i o n s of the t r a i n i n g objectives. Successful completion of the design requires the application of learning theory, the development and production of appropriate learning and evaluation materials, and the coordination of actions into an operational system which i s consistent with company practices. Starting with a set of t r a i n i n g objectives derived from the assessment of t r a i n i n g needs, c r i t e r i a are established by which success i n achieving objectives may be measured. Once the objectives are defined and the c r i t e r i a for success are established an appropriate sequence of learning a c t i v i t i e s within the formal i n s t r u c t i o n a l setting may be designed which w i l l r e s u l t i n the objectives being met by p a r t i c i p a n t s . The design of the i n s t r u c t i o n a l a c t i v i t i e s requires decisions with respect to factors such as s e l e c t i o n of the instructor; s e l e c t i o n of evaluation instruments; selection and sequencing of course content; selection of methods, techniques, and devices; production of i n s t r u c t i o n a l materials; p i l o t runs for lesson plans and revisions of the course; and selection and pre-testing of p a r t i c i p a n t s . In addition to the integration of theory and practice for instruction, the coordination of course schedules and locations within company operating procedures i s important. Legitimization The purpose of l e g i t i m i z a t i o n a c t i v i t i e s i s to negotiate the proposed program through the corporate r e s t r a i n t s of f i n a n c i a l budgeting and organizational climate. Any t r a i n i n g program which has been designed i s only one of many alternate courses of action by which company productivity may be improved. A management decision to proceed with the program depends upon i t s presentation as a worthwhile a c t i v i t y i n terms of return on investment. That presentation usually requires a f i n a n c i a l analysis of the program i n terms of fixed and variable costs, breakeven points, cash flows, contribution to p r o f i t s , and payback periods. In addition to the f i n a n c i a l aspects of a program, two other factors must be legitimized: (1) management acceptance and support of the program (2) organizational climate i n r e i n f o r c i n g the newly-learned behaviour on the job. Perhaps the most c r i t i c a l factor for a program's success i s the gaining of management commitment toward i t . (31) Strategies which re l a t e to the gaining of acceptance include the encouragement of management p a r t i c i p a t i o n i n the design of programs and the 34 provision of preview programs for management i n order to gain their approval of the proposed behavioural changes. Both the operational plans and the program content must be acceptable to management. Even more d i f f i c u l t to gain than management approval i s the willingness of employees throughout the company to accept and reinforce new behaviours which are taught i n the trai n i n g programs. The acceptance of the newly-learned behaviour i s more readi l y developed i f i t proceeds from top management downward, but the organizational climate must be thoroughly analyzed before dealing with the problem. Instructional Support Instruction within the formal i n s t r u c t i o n a l setting i s the f o c a l point of the program planning process. When participants i n the program meet formally for the f i r s t time with their i n s t r u c t i o n a l agent the majority of background planning tasks should be completed. There should be few planning duties to be performed during the i n s t r u c t i o n a l phase of the program but some a c t i v i t i e s are required for the support of i n s t r u c t i o n a l meetings. Before i n s t r u c t i o n commences a l l lesson plans, i n s t r u c t i o n a l materials, and evaluation instruments should be prepared and arrangements should be completed for any necessary tr a i n i n g rooms, audio-visual equipment, l i b r a r y resources, or supplies for part i c i p a n t s . The participants should be pre-tested and grouped for in s t r u c t i o n i f necessary, and the instructor and any special lecturers should be selected and prepared for the program. During the i n s t r u c t i o n a l a c t i v i t i e s some support services are required, so that the instructor has assistance available when required for various materials preparation or other similar projects which become necessary as the lessons progress. In actual practice many changes and decisions are required as i n s t r u c t i o n progresses and problems are encountered which were not considered i n the previous planning phases. The in s t r u c t i o n a l support functions are therefore very f l u i d and unpredictable. Instructional support tasks are performed when the need arises rather than by s p e c i f i c plan, and the important factor i n the program planning process i s to ensure that allowance i s made for such a set of indeterminate actions during the i n s t r u c t i o n a l phase. Evaluation Of a l l the functions of program planning, evaluation has received the most attention i n the l i t e r a t u r e . Evaluation i s useful i n j u s t i f y i n g the existence of tr a i n i n g i n industry through cost/benefit analysis, i n pinpointing and diagnosing problems and needs i n the organization, i n comparing the effectiveness of d i f f e r e n t t r a i n i n g methods, and i n determining the r e s u l t s of e f f o r t s which attempt to change behaviour. Two basic forms of evaluation are generally used i n business and industry: (1) process evaluation, which concerns the effectiveness of the tr a i n i n g system i t s e l f (2) product evaluation, which relates to the achievement of t r a i n i n g objectives by participants i n the program. Process evaluation of the program would be conducted through such a c t i v i t i e s as assessing the i n s t r u c t i o n a l presentation, comparing presently-used practices with those of other companies, contrasting alternative means of solving t r a i n i n g problems, and conducting cost/benefit analyses to estimate the value of the program. A l l of those practices would focus on the instructor, the e f f i c i e n c y of program planning, and other elements of the t r a i n i n g system. 36 The most commonly-used typology of product evaluation was devised by Kirkpatrick, and i t consists of four steps: (1) reaction - how well did participants l i k e the program? (2) learning - how much knowledge or s k i l l was acquired? (3) behaviour - what changes occurred i n on-the-job behaviour? (4) r e s u l t s - what changes occurred i n o v e r a l l company productivity or e f f i c i e n c y ? (33) Catelanello and Kirkpatrick reported that from 110 companies surveyed, 77 per cent evaluated programs with the reaction step, 50 per cent with the learning step, 54 per cent with the behaviour step, and 45 per cent with the res u l t s step. (16) Each of the steps requires a d i s t i n c t approach to evaluation. Reaction may be evaluated by questionnaires or interviews, learning by written or performance pre- and post-tests, and behaviour by pre- and post- on-the-job measures. Common evaluation techniques are not adequate to i s o l a t e the eff e c t s of tra i n i n g on long-term changes within a company. Evaluation procedures must be planned i n advance of the commencement of in s t r u c t i o n and must be related to given objectives of the course. Pre-testing and post-testing are desirable with respect to learning or behaviours which are the focus of the program, and continual evaluation throughout the program i s advantageous as well. In general, the in s t r u c t i o n and evaluation for a program should be planned and integrated as a unit. Maintenance of Behaviour A maintenance of behaviour function i s required i n conjunction with corporate t r a i n i n g programs to ensure that newly-learned behaviours w i l l be continued on the job. When employees return to the job from a formal t r a i n i n g program they frequently encounter an environment which w i l l not encourage their new behaviour, or else they may be unable to apply the behaviour i n the job s i t u a t i o n . The l a t t e r problem i s one of transfer of t r a i n i n g and i s most e a s i l y r e c t i f i e d by making the tr a i n i n g s i t u a t i o n as similar as i s possible to the job s i t u a t i o n . The problem of the work environment i s not so e a s i l y dealt with and frequently leads to f r u s t r a t i o n on th part of the returning p a r t i c i p a n t . Two fundamental aspects of the problem r e l a t e to reinforcement of learning and to analysi of the organizational climate. The employee's superior must be agreeable to the new behaviour so that he w i l l encourage i t and reinforce i t on the job while at the same time he w i l l not reinforce behaviours which c o n f l i c t with the new ones. Where the work environment w i l l not support the new behaviour steps may be taken to change the attitudes of other employees so that they w i l l accept the value of behaviour gained through t r a i n i n g programs. Steps within the t r a i n i n g program which may be planned as maintenance of behaviour a c t i v i t i e s include post program refresher courses, sequences of follow-up evaluations based on the desired behaviours, and separate programs which are designed to develop better corporate environments for dealing with change. CHAPTER THREE DEVELOPMENT OF THE INDEX OF PROGRAM PLANNING For two years, commencing i n July of 1971, a series of research procedures were conducted for the purpose of developing an instrument which would measure the standard of q u a l i t y present in a set of a c t i v i t i e s used by a company to plan and implement i t s management and supervisory t r a i n i n g programs. The study was based on the assumption that a standard of excellence for program planning might be established through the c o l l e c t i v e opinions of people who were well-informed on the subjects of program planning theory or management t r a i n i n g . I t was also based on the assumption that an integration of relevant theory, l i t e r a t u r e , and numerous opinions might be most e f f i c i e n t l y achieved through a series of structured exercises. Those exercises were based upon the model for program planning and were developed i n a sequential manner, with each exercise dependent upon the r e s u l t s of the one before for the content which was used. A t o t a l of three structured exercises were designed for the project. The exercises were designed to s o l i c i t the opinions of a variety of t r a i n i n g experts and s p e c i a l i s t s from within Canada. Approximately seventy people agreed to contribute their opinions for the study, and that group included representatives from u n i v e r s i t i e s , technical schools, community colleges, governments, and private industry. The majority of the group was resident within western Canada. No person was asked to p a r t i c i p a t e i n more than two of the three exercises, and no person was permitted to be involved i n both of the f i r s t two exercises. A l i s t of the participants and their positions i s provided i n Appendix D. 38 JUDGEMENTS OF STATEMENTS Following the development of the model and the d e f i n i t i o n of i t s constituent elements, as was outlined i n Chapter Two, the l i t e r a t u r e r e l a t i n g to t r a i n i n g was searched for i l l u s t r a t i o n s of corporate personnel development behaviours which were considered to be examples of the seven major program planning functions included i n the model. For each function, eight statements were constructed to specify behaviours which might be performed by a company i n the conduct of that function. Two separate exercises were then devised to test the appropriateness of the f i f t y - s i x statements. The f i r s t exercise was designed to determine i f the statements were considered to be v a l i d examples of the functions which they were intended to represent. The second exercise was designed to determine i f there was general agreement among judges with respect to the function i n the planning process of each behaviour as represented by the statement. In the f i r s t exercise, to determine the v a l i d i t y of statements, each of the seven planning functions was named, defined, and followed by the eight statements which had been created to i l l u s t r a t e behaviours conducted as part of the function. Training experts and s p e c i a l i s t s were then asked to rate each statement according to i t s v a l i d i t y as an example of the appropriate function. A five-choice scale was provided: (1) i n v a l i d example (2) poor example (3) adequate example (4) good example (5) excellent example. A copy of the f i r s t exercise, labelled "Judgements of Item V a l i d i t y " , may be seen i n Appendix A. That exercise was completed by eighteen judges. Any statement which did not receive a median score of 3.2 5 or more 40 on the v a l i d i t y scale of 1 to 5 was withdrawn because the judges did not consider i t to be a s u f f i c i e n t l y v a l i d example of the appropriate function. The f i f t y - s i x statements and their median scores which were derived from the f i r s t exercise may be seen in~Table I I I . As may be noted, the f i v e statements numbered 7, 9, 16, 38, and 53, were eliminated from further consideration because the judges did not consider them to be s u f f i c i e n t l y v a l i d examples. The o r i g i n a l data r e l a t i n g to exercise one may be found i n Appendix B. In the second exercise, which directed participants to categorize statements into functions, the f i f t y - s i x statements were l i s t e d i n random order. Judges were provided with the name and d e f i n i t i o n of each of the seven planning functions and were asked to categorize each statement i n turn into the function with which i t seemed to be most associated. A copy of the second exercise may be seen i n Appendix A. The second exercise was completed by sixteen judges, and any statement which was not categorized c o r r e c t l y by sixty per cent of the judges was eliminated because of a lack of agreement on the general purpose of the behaviour within the program planning process. The res u l t s of the second exercise may be seen i n Table III, and they indicate the percentage of judges who categorized each statement into i t s correct planning function. As may be noted, the six statements numbered 21, 24, 34, 48, 54, and 56, were categorized c o r r e c t l y by less than sixt y per cent of the judges, and were therefore eliminated from further consideration i n the study. Within the random order of the second exercise, the six corresponding statements were located i n positions 16, 1, 51, 3, 17, and 8. The raw data r e l a t i n g to exercise two may be found i n Appendix B. TABLE III RESULTS FOR VALIDITY AND CATEGORIZATION EXERCISES STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY (N=18) (N=16) CORPORATE ANALYSIS AND MANPOWER PLANNING 8. The o v e r a l l objectives of the company are 4.50 indicated i n a written document which i s kept up-to-date and which i s available to employees who are responsible for planning t r a i n i n g programs. The company i s engaged i n planning projects designed to estimate future company manpower requirements. The company attempts i n some formal manner to coordinate a l l h i r i n g , training, and promotions with studies of future manpower requirements. The company i s engaged i n projects designed to study and speculate on future roles and r e s p o n s i b i l i t i e s of i t s employees. 3.67 3.67 3.30 3.25 A. l i s t pf retirement dates of managers and supervisors i s maintained up-to-date by the company as part of i t s manpower planning operations. The company establishes plans for 5 to 10 years 4.00 i n the future and those plans are made known to employees who are responsible for planning t r a i n i n g programs. The company employs at least one person whose 2.30 primary function i s to study future developments i n society and to report on their significance for the company. The company i s engaged i n planning projects 3.67 designed to speculate, study, or recommend pot e n t i a l organizational changes or developments. 81.3 93.8 100.0 87.5 93.8 87.5 100.0 100.0 42 TABLE I I I (CONTINUED) STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY DETERMINING AND ASSESSING TRAINING NEEDS 9. A study was performed of employee morale, 2.50 75.0 e f f e c t i v e n e s s of communication, c l i m a t e f o r i n n o v a t i o n , degree of i n t e r n a l c o n f l i c t , or degree of commitment to c o r p o r a t e o b j e c t i v e s w i t h i n the company f o r the purpose of determining i f t r a i n i n g was necessary. 10. The work r e c o r d s of managers and s u p e r v i s o r s or 3.25 93.8 t h e i r subordinates were analyzed f o r such i n d i c a t o r s as waste m a t e r i a l s , r e j e c t s , re-runs, s a f e t y , s a l e s , or other measures of job performance to determine i f t r a i n i n g was necessary. 11. The p e r s o n n e l r e c o r d s of managers and 3.38 93.8 s u p e r v i s o r s or t h e i r subordinates were analyzed f o r such i n d i c a t o r s as absenteeism, l a t e n e s s , turnover, g r i e v a n c e s , demotions, or requests f o r t r a n s f e r f o r the purpose of determining i f t r a i n i n g was necessary. 12. Company managers and s u p e r v i s o r s were asked 3.39 93.8 f o r t h e i r o p i n i o n s as to the needs f o r t r a i n i n g i n the company. 13. An up-to-date company i n v e n t o r y of the 4.17 100.0 knowledge, s k i l l s , a t t i t u d e s , or i n t e r e s t s o f employees was used.to determine i f t r a i n i n g was necessary. 14. A job a n a l y s i s and/or task a n a l y s i s was 4.13 62 ..5 performed t o determine the necessary knowledge, a t t i t u d e s or s k i l l s r e q u i r e d f o r managerial and s u p e r v i s o r y jobs w i t h i n the company. 15. When problems were l o c a t e d i n company o p e r a t i o n s 3.83 93.8 a study was performed to assess the l i k e l y e f f e c t s of t r a i n i n g as a p o t e n t i a l a c t i o n to so l v e the problems. TABLE I I I (CONTINUED) 43 STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY 16. The on-the-job performance of managers and 3.17 93.8 s u p e r v i s o r s was r a t e d or judged t o determine i f t r a i n i n g was necessary. DESIGNING THE TRAINING PROGRAM 17. The i n s t r u c t i o n a l s i t u a t i o n was designed t o 4.50 87.5 simulate a c t u a l working c o n d i t i o n s as much as p o s s i b l e so t h a t behaviours l e a r n e d i n the program c o u l d e a s i l y be t r a n s f e r r e d to the work s i t u a t i o n . 18. A l l i n s t r u c t i o n a l m a t e r i a l s which were 3.50 87.5 necessary f o r the program were designed b e f o r e the program was f o r m a l l y begun. 19. The program i n s t r u c t o r was a c t i v e i n h e l p i n g 4.07 93.8 to d e s i g n the sequence of i n s t r u c t i o n a l events. 20. The program was designed s p e c i f i c a l l y f o r a 4.50 87.5 group of employees whose working s i t u a t i o n and t r a i n i n g needs were known to the program d e s i g n e r s . 21. The o b j e c t i v e s o f the t r a i n i n g program were 5.00 56.3 s t a t e d i n b e h a v i o u r a l terms which i n d i c a t e d the a p p r o p r i a t e behaviour to be learned, the standard o f performance t o be met, and the c o n d i t i o n s under which the performance was to take p l a c e . 22. Information was gathered to determine when 3.50 81.3 p a r t i c i p a n t s were r e l a t i v e l y f r e e from heavy work p r e s s u r e s and the t i m e t a b l e f o r the t r a i n i n g program was developed on the b a s i s o f t h a t i n f o r m a t i o n . 23. Lesson plans or meeting procedures were put 3.50 75.0 through p i l o t t e s t s and were r e v i s e d wherever necessary b e f o r e the program was f o r m a l l y begun. TABLE III (CONTINUED) 44 STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY 24. The instruments and devices for the assessment 3.70 18.8 of the program and i t s participants were designed before the program was formally begun. LEGITIMIZATION 25. A series of cost/benefit studies of alternative 4.25 87.5 approaches to the tr a i n i n g program was used to convince management of the e f f i c i e n t use of funds by the proposed tr a i n i n g programs. 26. Approval for the program was f a c i l i t a t e d by 3.83 62.5 giving management an opportunity at d i f f e r e n t points i n the planning process to suggest changes which would bring the program more into l i n e with company objectives and expectations. 27. A separate budget for the tr a i n i n g program 3.79 87.5 was exhibited for management approval. 2 8. Short preview programs were provided for the 3.30 81.3 approval of top management before the f u l l scale program was begun. 29. Management demands for cost controls were 3.50 87.5 part l y met by c a l c u l a t i n g both a variable and a fixed cost component for the training.program. 30. A formal presentation of the program proposal 4.50 81.3 was offered to management i n which the values of the t r a i n i n g for the company were c l e a r l y outlined. 31. A return on investment was calculated for the 3.70 75.0 program i n order to convince management that the program was of value to the company. 32. To f a c i l i t a t e program approval, management 4.17 75.0 was asked to pa r t i c i p a t e i n the development of the program. TABLE III (CONTINUED) 45 STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY INSTRUCTIONAL SUPPORT 33. When s p e c i f i c resource people from within the 4.50 81.3 company were requested for the tr a i n i n g program by the instructor the company attempted to obtain them and make arrangements for their release from other duties. 34. Adequate reference materials r e l a t i n g to the 5.00 56.3 program of i n s t r u c t i o n were available i n s u f f i c i e n t quantities for a l l p a r t i c i p a n t s . 35. The instructor was provided with s u f f i c i e n t 4.50 87.5 s e c r e t a r i a l assistance to meet h i s requirements before, during, and after the i n s t r u c t i o n a l phase of the tr a i n i n g program. 36. A l l necessary audio-visual equipment needed 5.00 75.0 for the program was provided by the company either through r e n t a l or ownership. 37. The tra i n i n g program was conducted i n an 5.00 87.5 area which was s p e c i f i c a l l y allocated to the program by the company, and i n which no other company a c t i v i t i e s were allowed to intrude during the conduct of the program. 38. Any spe c i a l arrangements for food services 3.10 93.8 at coffee breaks, luncheon meetings, or dinner meetings were handled by company employees other than the inst r u c t o r . 39. Any necessary arrangements or bookkeeping 3.2 5 93.8 d e t a i l s for f i e l d t r i p s , s pecial meetings, conferences, etc. were handled by company employees other than the instructor, leaving the instructor free of such problems. 40. At least one member of the planning group for 3.94 81.3 the program assisted the instructor throughout the i n s t r u c t i o n a l phase i n making any necessary adjustments i n the tra i n i n g program. TABLE III (CONTINUED) 46 STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY EVALUATION 41. Participants were evaluated at several stages 3.93 81.3 in the program to determine th e i r progress toward achieving course objectives. 42. The i n s t r u c t i o n a l a b i l i t y of the instructor or group leader was formally evaluated on the basi of h i s performance during the program. 43. The objectives of the program were used as a basis from which to evaluate the learning and achievements of the part i c i p a n t s . 44. Before and after measures were made among the participants and among a sim i l a r group of employees not being trained with respect to objectives of the program for the purpose of determining the ef f e c t s of the program. 45. The on-the-job behaviour of participants was observed or measured both before and after the program to evaluate the e f f e c t s of the trai n i n g program. 46. Participants were tested before the program was formally begun to determine their s t a r t i n g levels of knowledge, s k i l l s , or attitudes which were relevant to the course objectives, so that the e f f e c t s of the program could be evaluated more r e a d i l y . 47. The planning procedures which were used to develop the program were evaluated to determine i f improvements could have been made i n the planning processes. 48. A systematic schedule of evaluations was planned 3.93 31.3 for use at interv a l s after the program was completed as a means of checking on the retention levels of newly-learned behaviours over periods of time. 3.50 87.5 s 4.50 100.0 5.00 100.0 5.00 93.8 3.83 68.8 3.83 68.8 TABLE III (CONTINUED) 47 STATEMENT MEDIAN PER CENT VALIDITY CATEGORIZED SCORE CORRECTLY MAINTENANCE OF BEHAVIOUR 49. Short refresher programs were provided for 3.50 participants at interv a l s after the o r i g i n a l program was completed. 50. The superior of each participant was requested 4.17 to attend a short informative program which outlined the behaviours to be learned by program participants, so that the superior was aware of what to expect from h i s subordinates on-the-job. 51. Some form of personal counseling was provided 4.00 for participants who experienced d i f f i c u l t y i n transferring the tr a i n i n g behaviours to the job setting after the program was completed. 52. The superior of each participant was requested 3.79 to p o s i t i v e l y reinforce the newly-learned behaviour on-the-job. 53. Top management issued instructions that 2.10 behaviours learned i n the t r a i n i n g program were expected to be used on-the-job after. the completion of the program. 54. The environment within which the participants 3.50 worked was analyzed to determine i f the newly-learned behaviours would be encouraged and supported so that those behaviours would continue to be used on-the-job. 55. Participants who had problems i n implementing 3.90 the newly-learned behaviours on-the-job were provided with special additional t r a i n i n g sessions. 56. The superior of each program participant was requested to submit a report to either the trai n i n g s p e c i a l i s t or to top management out l i n i n g the extent to which the newly-learned behaviours were used on-the-job by the participant after the completion of the program. 3.70 93.8 68.8 93.8 93.8 62.5 50.0 87.5 56.3 48 SELECTION OF STATEMENTS The objective of the f i r s t two exercises was to eliminate from further consideration, any statement which was not considered to be a good example of i t s appropriate function or which lacked c l a r i t y with respect to i t s general purpose i n the o v e r a l l planning procedure. Of the f i f t y - s i x statements from the o r i g i n a l l i s t , f o r t y - f i v e met the c r i t e r i a for both of the f i r s t two exercises, and those statements were included i n a t h i r d exercise. The objective of the t h i r d exercise was to estimate the importance of each stated behaviour i n the program planning process for business and i n d u s t r i a l training, and to select a number of statements for i n c l u s i o n i n the Index of Program Planning on the basis of the degree of agreement among the judges with respect to the importance of the behaviours. Since agreement was possible at any l e v e l of importance, both the highly important behaviours as well as behaviours of l i t t l e importance were available for selection with equal l i k e l i h o o d . For the t h i r d exercise, a l i s t of the f o r t y - f i v e statements was provided, ordered according to function but with no i d e n t i f y i n g labels for the respective functions. A scale of importance was also provided, with 1 indicating that the statement was of no importance i n planning management tr a i n i n g programs, and 9 indicating that the behaviour as stated was extremely important i n planning programs. Each p a r t i c i p a t i n g judge was asked to indicate his opinion with respect to the importance of each behaviour, using the scale of importance which was provided. A copy of the t h i r d exercise may be seen i n Appendix A. 49 Exercise three was completed by f i f t y - e i g h t judges, and the frequency d i s t r i b u t i o n s which were derived from the opinions of those judges may be seen i n Appendix C. The responses of the judges indicated that the average importance of each of the f o r t y - f i v e statements was greater than f i v e on a nine-point scale. However, the lack of agreement among judges was also evident, as the range of opinion covered a l l nine points on the scale for twenty-nine statements, and eight points on the scale for a further thirteen statements. The instructions which accompanied exercise three did not specify that the nine categories were to be considered of equal width, and the judges were therefore able to interpret the nine-point scale i n many d i f f e r e n t ways. The category widths may have been interpreted i n d i f f e r e n t terms among d i f f e r e n t judges, and the widths of categories at d i f f e r e n t positions on the scale may have been considered unequal by an i n d i v i d u a l judge. In consideration of those possible i n e q u a l i t i e s , the o r i g i n a l frequency d i s t r i b u t i o n s were considered unsuitable for s t a t i s t i c a l analysis, and the o r i g i n a l data was used to transform a l l forty-f i v e d i s t r i b u t i o n s onto a common underlying continuum for which the i n t e r v a l widths were calculated and known, even though they may not have been of equal s i z e . The construction of an underlying continuum and the transformation of the f o r t y - f i v e d i s t r i b u t i o n s onto i t was completed using the Method of Successive Intervals, as outlined by Edwards i n Techniques of Attitude Scale Construction. (26) The method required that the o r i g i n a l frequency d i s t r i b u t i o n for each statement be converted to a proportional d i s t r i b u t i o n , and that the proportional d i s t r i b u t i o n be successively summed 50 across the categories from l e f t to r i g h t to produce a cumulative proportion d i s t r i b u t i o n . At that point, an assumption was made that the cumulative proportion d i s t r i b u t i o n s were normal for each statement when they were projected onto the common continuum, which allowed the unit-normal curve and i t s well-known s t a t i s t i c a l properties to be used i n the development of the continuum. The z scores were then derived from the table of the unit-normal curve for each of the cumulative proportions exhibited across the categories for each of the f o r t y - f i v e statements. The differences i n terms of z scores were then calculated between each successive i n t e r v a l across the d i s t r i b u t i o n s . In some instances the lower category inter v a l s did not include z scores since the d i s t r i b u t i o n commenced at a higher l e v e l on the scale, and therefore, no differences i n z score was calculated. For each column of category interva l s , an average difference with respect to z scores was calculated for the t o t a l number of z score differences which were included i n the column. Those average widths i n terms of z scores were then summed successively from l e f t to r i g h t to produce the common underlying continuum which was the objective of the procedure. A number of tables which i l l u s t r a t e the transformation procedures i n d e t a i l may be found i n Appendix C. For purposes of selecting statements for the Index of Program Planning, not a l l points on the o r i g i n a l frequency d i s t r i b u t i o n s were transformed onto the underlying continuum which was constructed. A comparison of a l l statements was possible on a common basis using only the median value of importance and the semi-interquartile range from each of the f o r t y - f i v e d i s t r i b u t i o n s on the common continuum. Because the 51 median value was located at the f i f t i e t h percentile i n the di s t r i b u t i o n , and the semi-interquartile range was the difference i n scale values between the seventy-fifth and twenty-fifth percentiles, only three values were required for each d i s t r i b u t i o n on the common continuum: the three percentile locations, Por., 2. b P,.^ , and P_^. Each of those values was calculated by determining 50 75 i t s location on the o r i g i n a l frequency d i s t r i b u t i o n , and then locating the value of that point on the common underlying continuum. Since the development of the underlying continuum and the subsequent transformation of the o r i g i n a l d i s t r i b u t i o n s onto i t used the unit-normal curve as a s t a t i s t i c a l foundation, the components of that curve were used to complete the necessary analysis for the selection of statements for the Index of Program Planning. The analysis of agreement among judges with respect to each statement was based upon the semi-interquartile range, commonly la b e l l e d "Q", as the primary s t a t i s t i c . In the unit-normal curve the value of Q i s 1.36, and since that curve was the foundation of the f o r t y - f i v e transformed d i s t r i b u t i o n s , the value of 1.36 for Q was set as the standard for acceptance or re j e c t i o n of statements for the Index of Program Planning. In effect, any statement which had a value for Q of more than 1.36 exhibited more disagreement among judges than would have been expected had the d i s t r i b u t i o n s of opinions been s t a t i s t i c a l l y normal. Any statement with a value for Q of 1.36 or less was included i n the Index of Program Planning. The f o r t y - f i v e statements which were included i n exercise three, along with their associated Q values and median values of importance on the common underlying continuum may be seen i n Table IV. As may be noted, twenty-six statements were selected for the Index of 52 Program Planning on the basis of their Q values being 1.36 or less. The median values of importance for each of the chosen statements indicated the c o l l e c t i v e opinions of the judges with respect to the importance of each behaviour i n the planning of management tra i n i n g programs, and those values were a major element within the Index of Program Planning. THE INDEX OF PROGRAM PLANNING The major objective of th i s study was to develop an instrument which might be used to measure the standard of q u a l i t y with respect to program planning procedures for managerial and supervisory t r a i n i n g programs i n business and industry. Each phase i n the study, including the development of a model of program planning functions, the selection of statements to represent the seven major program planning functions i n the model, and the testing of those statements i n three separate exercises, was a contributing factor i n the development of the measurement instrument. As a r e s u l t of the sequence outlined above, an Index of Program Planning was constructed for the purpose of measuring standards of qua l i t y for program planning. The compilation of i n d i v i d u a l statements into a measurement index was based upon three assumptions: (1) better q u a l i t y program planning would be indicated by the use of more of the seven major functions i n the planning model (2) better quality program planning would be indicated by the use of a greater number of d i f f e r e n t behaviours i n the conduct of any p a r t i c u l a r planning function (3) better q u a l i t y program planning would be indicated by the use of more important behaviours rather than less important behaviours i n the conduct of any planning function. TABLE IV VALUES OF Q AND MEDIAN VALUES OF IMPORTANCE FOR EXERCISE THREE STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE CORPORATE ANALYSIS AND MANPOWER PLANNING 1. The o v e r a l l o b j e c t i v e s o f the company are 1.69 2.87 i n d i c a t e d i n a w r i t t e n document which i s kept up-to-date and which i s a v a i l a b l e to employees who are r e s p o n s i b l e f o r p l a n n i n g t r a i n i n g programs. 2. The company i s engaged i n p l a n n i n g p r o j e c t s 1.42 2.10 designed to estimate f u t u r e company manpower requirements. 3. The company attempts i n some formal manner 1.18 1.94 to c o o r d i n a t e a l l h i r i n g , t r a i n i n g , and promotions with s t u d i e s o f f u t u r e manpower requirements. 4. The company i s engaged i n p r o j e c t s designed 1.04 1.68 to study and s p e c u l a t e on f u t u r e r o l e s and r e s p o n s i b i l i t i e s of i t s employees. 5. A l i s t o f r e t i r e m e n t dates o f managers and 1.42 1.37 s u p e r v i s o r s i s maintained up-to-date by the company as p a r t o f i t s manpower p l a n n i n g o p e r a t i o n s . 6. The company e s t a b l i s h e s plans f o r 5 to 10 1.36 2.00 years i n the f u t u r e and those plans are made known to employees who are r e s p o n s i b l e f o r p l a n n i n g t r a i n i n g programs. 7. The company i s engaged i n p l a n n i n g p r o j e c t s 1.2 3 1.53 designed to sp e c u l a t e , study, or recommend p o t e n t i a l o r g a n i z a t i o n a l changes or development s. 54 TABLE IV (CONTINUED) STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE DETERMINING AND ASSESSING TRAINING NEEDS 8. The work r e c o r d s o f managers and s u p e r v i s o r s 1.08 1.85 or t h e i r subordinates were analyzed f o r such i n d i c a t o r s as waste m a t e r i a l s , r e j e c t s , re-runs, s a f e t y , s a l e s , or other measures o f job performance to determine i f t r a i n i n g was necessary. 9. The pe r s o n n e l r e c o r d s of managers and 1.27 1.36 s u p e r v i s o r s or t h e i r s ubordinates were analyzed fo r such i n d i c a t o r s as absenteeism, l a t e n e s s , turnover, g r i e v a n c e s , demotions, or requests f o r t r a n s f e r f o r the purpose o f determining i f t r a i n i n g was necessary. 10. Company managers and s u p e r v i s o r s were asked 1.44 2.39 for t h e i r o p i n i o n s as to the needs f o r t r a i n i n g i n the company. 11. An up-to-date company i n v e n t o r y o f the knowledge, s k i l l s , a t t i t u d e s , or i n t e r e s t s o f employees was used to determine i f t r a i n i n g was necessary. 12. A job a n a l y s i s and/or task a n a l y s i s was performed t o determine the necessary knowledge, a t t i t u d e s , or s k i l l s r e q u i r e d f o r managerial and s u p e r v i s o r y jobs w i t h i n the company. 13. When problems were l o c a t e d i n company 1.41 1.91 o p e r a t i o n s a study was performed to assess the l i k e l y e f f e c t s o f t r a i n i n g as a p o t e n t i a l a c t i o n to so l v e the problems. 1.45 1.92 1.46 2.34 TABLE IV (CONTINUED) 55 STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE DESIGNING THE TRAINING PROGRAM 14. The i n s t r u c t i o n a l s i t u a t i o n was designed to 1.19 2.44 simulate actual working conditions as much as possible so that behaviours learned i n the program could e a s i l y be transferred to the work s i t u a t i o n . 15. A l l i n s t r u c t i o n a l materials which were 1.63 1.72 necessary for the program were designed before the program was formally begun. 16. The program instructor was active i n helping 1.25 2.42 to design the sequence of i n s t r u c t i o n a l events. 17. The program was designed s p e c i f i c a l l y for a 1.08 2.58 group of employees whose working s i t u a t i o n and tra i n i n g needs were known to the program designers. 18. Information was gathered to determine when 1.38 1.96 participants were r e l a t i v e l y free from heavy work pressures and the timetable for the tra i n i n g program was developed on the basis of that information. 19. Lesson plans or meeting procedures were put 1.21 1.59 through p i l o t tests and were revised wherever necessary before the program was formally begun. LEGITIMIZATION 20. A series of cost/benefit studies of alternative 1.28 1.41 approaches to the tra i n i n g program were used to convince management of the e f f i c i e n t use of funds by the proposed t r a i n i n g program. TABLE IV (CONTINUED) STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE 21. Approval for the program was f a c i l i t a t e d by 1.43 2.12 giving management an opportunity at d i f f e r e n t points i n the planning process to suggest changes which would bring the program more into l i n e with company objectives and expectations. 22. A separate budget for the tr a i n i n g program 1.50 2.00 was exhibited for management approval. 2 3. Short preview programs were provided for the 1.2 5 1.42 approval of top management before the f u l l scale program was begun. 24. Management demands for cost controls were 1.16 1.37 partly met by ca l c u l a t i n g both a variable and a fixed cost component for the tr a i n i n g program. 25. A formal presentation of the program proposal 1.41 2.24 was offered to management i n which the values of the t r a i n i n g for the company were c l e a r l y outlined. 26. A return on investment was calculated for the 1.29 1.23 program i n order to convince management that the program was of value to the company. 27. To f a c i l i t a t e program approval, management was 1.2 7 1.97 asked to pa r t i c i p a t e i n the development of the program. INSTRUCTIONAL SUPPORT 28. When s p e c i f i c resource people from within the 1.28 2.23 company were requested for the tra i n i n g program by the instructor the company attempted to obtain them and make arrangements for the i r release from other duties. 29. The instructor was provided with s u f f i c i e n t 1.18 1.83 s e c r e t a r i a l assistance to meet h i s requirements before, during, and after the i n s t r u c t i o n a l phase of the t r a i n i n g program. TABLE IV (CONTINUED) STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE 30. A l l necessary a u d i o - v i s u a l equipment needed 1.20 2.20 f o r the program was pr o v i d e d by the company e i t h e r through r e n t a l or ownership. 31. The t r a i n i n g program was conducted i n an area 1.35 2.71 which was s p e c i f i c a l l y a l l o c a t e d to the program by the company, and i n which no other company a c t i v i t i e s were allowed t o i n t r u d e d u r i n g the conduct o f the program. 32. Any necessary arrangements or bookkeeping 1.45 1.47 d e t a i l s f o r f i e l d t r i p s , s p e c i a l meetings, conferences, e t c . were handled by company employees other than the i n s t r u c t o r , l e a v i n g the i n s t r u c t o r f r e e o f such problems. 33. At l e a s t one member of the p l a n n i n g group f o r 1.16 1.65 the program a s s i s t e d the i n s t r u c t o r throughout the i n s t r u c t i o n a l phase i n making any necessary adjustments i n the t r a i n i n g program. EVALUATION 34. P a r t i c i p a n t s were e v a l u a t e d at s e v e r a l stages 1.22 2.17 i n the program to determine t h e i r progress toward a c h i e v i n g course o b j e c t i v e s . 35. The i n s t r u c t i o n a l a b i l i t y o f the i n s t r u c t o r 1.57 2.18 or group leader was f o r m a l l y e v a l u a t e d on the b a s i s o f h i s performance d u r i n g the program. 36. The o b j e c t i v e s o f the program were used as a 1.17 2.69 b a s i s from which t o e v a l u a t e the l e a r n i n g and achievements o f the p a r t i c i p a n t s . 37. Before and a f t e r measures were made among the 1.11 1.94 p a r t i c i p a n t s and among a s i m i l a r group of employees not b e i n g t r a i n e d w i t h r e s p e c t to o b j e c t i v e s o f the program f o r the purpose o f determining the e f f e c t s o f the program. TABLE IV (CONTINUED) 58 STATEMENT Q MEDIAN VALUE VALUE OF IMPORTANCE 38. The on-the-job behaviour of participants was 1.19 2.44 observed or measured both before and after the program to evaluate the e f f e c t s of the t r a i n i n g program. 39. Participants were tested before the program 1.41 1.93 was formally begun to determine their s t a r t i n g levels of knowledge, s k i l l s , or attitudes which were relevant to the course objectives, so that the e f f e c t s of the program could be evaluated more r e a d i l y . 40. The planning procedures which were used to 1.05 1.87 develop the program were evaluated to determine i f improvements could have been made i n the planning processes. MAINTENANCE OF BEHAVIOUR 41. Short refresher programs were provided for 1.26 2.01 participants at inte r v a l s after the o r i g i n a l program was completed. 42. The superior of each participant was requested 1.58 2.49 to attend a short informative program which outlined the behaviours to be learned by program participants, so that the superior was aware of what to expect from h i s subordinates on-the-job. 43. The superior of each pa r t i c i p a n t was requested 1.38 2.54 to p o s i t i v e l y reinforce the newly-learned behaviour on-the-job. 44. Participants who had problems i n implementing 1.44 1.95 the newly-learned behaviours on-the-job were provided with special additional t r a i n i n g sessions. 45. Some form of personal counseling was provided 1.45 2.34 for participants who experienced d i f f i c u l t y i n transferring the tr a i n i n g behaviours to the job setting after the program was completed. 59 The implications of those assumptions were that the Index of Program Planning would be constructed by as many statements as possible from each of the seven major functions, that each of the major functions would be represented i n the index, and that a procedure would be established to evaluate numerically the r e l a t i v e importance of the s p e c i f i c behaviours used by a company in i t s program planning procedures. In addition, i t was necessary that those requirements for the construction of the Index of Program Planning would be met using only those statements which met the following c r i t e r i a : (1) the statement was a v a l i d example of the function i t represented (2) the function of the statement i n the program planning process was generally agreed upon by t r a i n i n g experts and s p e c i a l i s t s (3) the importance of the statement i n the program planning process was generally agreed upon by t r a i n i n g experts and s p e c i a l i s t s . The testing of a l l statements included i n the Index of Program Planning, with respect to those three c r i t e r i a , was accomplished by means of the three exercises which were designed for the study. The Index of Program Planning was developed i n consideration of the constraints which have been l i s t e d , and the instrument may be seen i n Table V. I t consists of twenty-six statements selected from the o r i g i n a l f i f t y - s i x , and each of the major planning functions i s represented. However, because the statements were chosen without reference to s p e c i f i c functions, there i s a v a r i a t i o n i n the number of statements representing each function. The statements were l i s t e d sequentially with respect to the functions which were represented, so that the f i r s t four statements represented the function Corporate Analysis 60 and Manpower Planning, the following two statements represented the function Determining and Assessing Training Needs, statements seven to ten represented Designing the Training Program, statements eleven to f i f t e e n represented the Legitimization function, statements sixteen to twenty represented the function Instructional Support, statements twenty-one to twenty-five represented Evaluation, and statement twenty-six represented the Maintenance of Behaviour function. A summary of the statement numbers and the functions represented may be found i n Table VI. The Index of Program Planning was designed to calculate a numerical value which would represent a cumulative i n d i c a t i o n of the standard of qual i t y exhibited by a company i n i t s program planning procedures. That numerical value was derived, using the index, by combining the values of importance and the r e l a t i v e percentage of use by a company i n i t s planning procedures with respect to each of the twenty-six behaviours l i s t e d i n the index. S p e c i f i c a l l y , the Index of Program Planning was designed for use i n the following manner: (1) for each of the twenty-six statements, a company i s requested to determine the percentage of times that behaviour was used i n the planning procedures for a l l of the management and supervisory tr a i n i n g programs which were planned and implemented by the company over a period of twenty-four months (2) for each of the statements the percentage of use as was determined for that behaviour i s multi p l i e d by the value of importance for the statement (3) the twenty-six products r e s u l t i n g from the mu l t i p l i c a t i o n s are added together to create a t o t a l sum, which would be the numerical value representing the standard of program planning qu a l i t y exhibited by the company. 61 TABLE V THE INDEX OF PROGRAM PLANNING STATEMENT MEDIAN PER PRODUCT IMPORTANCE CENT USAGE 1. The company attempts i n some formal 19.4 manner to coordinate a l l h i r i n g , t raining, and promotions with studies of future manpower requirements. 2. The company i s engaged i n projects 16.8 designed to study and speculate on future roles and r e s p o n s i b i l i t i e s of i t s employees. 3. The company establishes plans for 5 2 0.0 to 10 years i n the future and those plans are made known to employees who are responsible for planning t r a i n i n g programs. 4. The company i s engaged i n planning 15.3 projects designed to speculate, study, or recommend po t e n t i a l organizational changes or developments. 5. The work records of managers and 18.5 supervisors or their subordinates were analyzed for such indicators as waste materials, rejects, re-runs, safety, sales, or other measures of job performance to determine i f trai n i n g was necessary. 6. The personnel records of managers 13.6 and supervisors or their subordinates were analyzed for such indicators as absenteeism, lateness, turnover, grievances, demotions, or requests for transfer for the purpose of determining i f t r a i n i n g was necessary. 1. The values of median importance used within the index are the median values of importance for d i s t r i b u t i o n s on the common continuum mult i p l i e d by a factor of ten. 62 TABLE V (CONTINUED) STATEMENT MEDIAN PER PRODUCT IMPORTANCE CENT USAGE  7. The i n s t r u c t i o n a l s i t u a t i o n was 24.4 designed to simulate actual working conditions as much as possible so that behaviours learned i n the program could e a s i l y be transferred to the work s i t u a t i o n . 8. The program instructor was active 24.2 i n helping to design the sequence of i n s t r u c t i o n a l events. 9. The program was designed s p e c i f i c a l l y 25.8 for a group of employees whose working s i t u a t i o n and t r a i n i n g needs were known to the program designers. 10. Lesson plans or meeting procedures were put through p i l o t tests and were revised wherever necessary before the program was formally begun. 11. A series of cost/benefit studies of alternative approaches to the t r a i n i n g program was used to convince management of the e f f i c i e n t use of funds by the proposed t r a i n i n g program. 12. Short preview programs were provided for the approval of top management before the f u l l scale program was begun. 13. Management demands for cost controls 13.7 were partly met by c a l c u l a t i n g both a variable and a fixed cost component for the t r a i n i n g program. 14. A return on investment was calculated 12.3 for the program i n order to convince management that the program was of value to the company. 15.9 14.1 14.2 63 TABLE V (CONTINUED) STATEMENT MEDIAN PER PRODUCT IMPORTANCE CENT USAGE 15. To f a c i l i t a t e program approval, 19.7 management was asked to pa r t i c i p a t e i n the development of the program. 16. When s p e c i f i c resource people from 22.3 within the company were requested for the t r a i n i n g program by the instructor the company attempted to obtain them and make arrangements for their release from other duties. 17. The instructor was provided with 18.3 s u f f i c i e n t s e c r e t a r i a l assistance to meet h i s requirements before, during, and after the i n s t r u c t i o n a l phase of the tr a i n i n g program. 18. A l l necessary audio-visual equipment 22.0 needed for the program was provided by the company either through r e n t a l or ownership. 19. The training program was conducted i n 2 7.1 an area which was s p e c i f i c a l l y allocated to the program by the company, and i n which no other company a c t i v i t i e s were allowed to intrude during the conduct of the program. 20. At least one member of the planning 16.5 group for the program assisted the instructor throughout the i n s t r u c t i o n a l phase i n making any necessary adjustments i n the tr a i n i n g program. 21. Participants were evaluated at several 21.7 stages i n the program to determine their progress toward achieving course objectives. 64 TABLE V (CONTINUED) STATEMENT MEDIAN PER PRODUCT IMPORTANCE CENT USAGE 22. The objectives of the program were 26.9 used as a basis from which to evaluate the learning and achievements of the parti c i p a n t s . 23. Before and after measures were made 19.4 among the participants and among a similar group of employees not being trained with respect to objectives of the program for the purpose of determining the eff e c t s of the program. 24. The on-the-job behaviour of participants 24.4 was observed or measured both before and after the program to evaluate the effe c t s of the tra i n i n g program. 25. The planning procedures which were used 18.7 to develop the program were evaluated to determine if.improvements could have been made i n the planning processes. 26. Short refresher programs were provided 2 0.1 for participants at interv a l s after the o r i g i n a l program was completed. TOTAL SCORE 65 TABLE VI RELATIONSHIP OF FUNCTIONS AND STATEMENTS IN THE INDEX OF PROGRAM PLANNING PLANNING FUNCTION INDEX STATEMENT NUMBERS Corporate Analysis and Manpower Planning 1, 2, 3, 4 Determining and Assessing Training Needs 5, 6 Designing the Training Program 7, 8, 9, 10 Legitimization 11, 12, 13, 14, 15 Instructional Support 16, 17, 18, 19, 2 0 Evaluation 21, 22, 23, 24, 25 Maintenance of Behaviour 26 If a company were to use each of the twenty-six stated behaviours one hundred per cent of the time when planning and implementing i t s t r a i n i n g programs for managers and supervisors, the value of 505.3 would be calculated from the index, and that i s the highest score possible. I f a company were to use some of the behaviours only a portion of the time when programs were planned, a score would be calculated i n the range of 0 to 505.3, depending upon the s p e c i f i c combination of the r e l a t i v e importance of the behaviours which were used, and the percentage of time each behaviour was included i n the planning process. In general, the scores would be greater i n d i r e c t proportion to a combination of the number of behaviours used, the r e l a t i v e l y greater importance of the values for those behaviours used, and the r e l a t i v e l y greater percentage of time for which the behaviours were used. CHAPTER FOUR ANALYSIS AND VALIDATION OF THE INDEX OF PROGRAM PLANNING In conjunction with the development of the Index of Program Planning, a number of analyses were conducted for the purpose of testing some of the factors associated with the formation and use of the index. Those analyses included tests related to the developmental procedures for the index, and to estimates of r e l i a b i l i t y and v a l i d i t y for the index. TESTS OF DEVELOPMENTAL PROCEDURES Two analyses were conducted with respect to the procedures which were used i n the development of the Index of Program Planning. The f i r s t test was associated with the transformation of the o r i g i n a l data from exercise three onto the underlying continuum, and the second was related to the r e l i a b i l i t y of opinions expressed by the group of judges who participated i n the study. Method of Successive Intervals The method of successive inte r v a l s was used to create an underlying common continuum with respect to the f o r t y - f i v e d i s t r i b u t i o n s of opinion i n exercise three, and to transform the o r i g i n a l data onto that t h e o r e t i c a l continuum. A necessary assumption for the use of that procedure was that the d i s t r i b u t i o n s of opinion were d i s t r i b u t e d normally when projected onto the common continuum. In order to check whether that assumption was appropriate for the observed data, a test for i n t e r n a l consistency was applied. The test, as outlined by Edwards, compared the discrepancies between the o r i g i n a l empirical d i s t r i b u t i o n s and 66 the t h e o r e t i c a l l y derived d i s t r i b u t i o n s . (26) The degree of error was determined by reconstructing the o r i g i n a l set of data, using only the information available within the common underlying scale and the t h e o r e t i c a l d i s t r i b u t i o n s accompanying i t . Edwards reported that an average discrepancy i n the range of 2.4 per cent was t y p i c a l , and indicated that the empirical data was appropriate with respect to the assumption of normality when the discrepancies were within that range. (26:138) The average deviation found i n the present study was 3.0 per cent. I t was therefore concluded that the assumptions of normality were acceptable and that the data were appropriate for the transformation procedures. R e l i a b i l i t y of Opinions A second analysis was conducted with respect to the Index of Program Planning, to estimate the possible e f f e c t s upon the index had a d i f f e r e n t group of judges been used i n the development phases. For purposes of that analysis, the group of f i f t y - e i g h t judges was randomly sorted into two separate groups, using the toss of a coin as the c r i t e r i a for group selec t i o n . As a r e s u l t of the selection procedure, the f i r s t random group consisted of twenty-three judges, and the second group included t h i r t y -f i v e judges. The two groups were then compared with respect to the median values of importance for the f o r t y - f i v e statements i n the t h i r d exercise, had each group been i n d i v i d u a l l y included i n the development of the index. The c o r r e l a t i o n between the two random groups was considered to be an estimate of the c o r r e l a t i o n which might have existed between the actual group of p a r t i c i p a t i n g judges and any other group of judges which 68 might have been chosen. The value of the c o r r e l a t i o n was +0.84 and that s t a t i s t i c was s i g n i f i c a n t beyond the .001 l e v e l of sig n i f i c a n c e . I t was therefore concluded that another set of judges would l i k e l y have contributed a similar set of median values of importance for the f o r t y - f i v e statements. The two random groups were also compared with respect to the degree of agreement exhibited toward the importance of the f o r t y - f i v e statements included i n the t h i r d exercise. The disagreement among judges within the t o t a l group of f i f t y - e i g h t was evident, and the c o r r e l a t i o n between the two random groups which were selected from the t o t a l was found to be +0.05. I t was therefore concluded that another set of judges might e a s i l y have selected another set of statements out of the o r i g i n a l f o r t y - f i v e for i n c l u s i o n i n the Index of Program Planning, as the s t a t i s t i c was not s i g n i f i c a n t even at the .25 l e v e l of s i g n i f i c a n c e . Further testing was then conducted to determine which s p e c i f i c statements might have been.selected for the index by each of the random groups. The f i r s t group would have selected twenty-four statements according to the c r i t e r i a which were used for the se l e c t i o n of statements by the t o t a l group of judges, and the second group would have chosen twenty-seven statements. The two random groups would have selected fourteen statements i n common. Those fourteen statements are included i n the Index of Program Planning as numbers 2, 4, 7, 8, 9, 10, 12, 13, 14, 17, 18, 22, 23, and 25. A summary of the s p e c i f i c statements which were selected by the two random groups of judges may be viewed i n Appendix C. SURVEY OF COMPANIES In an attempt to estimate the r e l i a b i l i t y and v a l i d i t y of the Index of Program Planning, an opportunity was provided for i t s repeated use i n a survey of companies. A panel of one hundred companies from within the greater Vancouver area of B r i t i s h Columbia was selected for the survey from those companies l i s t e d i n the 1968 Canada Manpower Centre Area P r o f i l e for  Metropolitan Vancouver (15) or the 1971 B r i t i s h Columbia Trade  Directory. (10) In i d e n t i f y i n g appropriate firms for the panel, the following c r i t e r i a were considered: (1) the company was engaged i n business within the greater Vancouver area, including Vancouver, West Vancouver, North Vancouver, Burnaby, or Richmond (2) the company was l i s t e d as employing two hundred people or more (3) the company, was not a subsidiary of another firm i n the panel. Approximately one hundred and f i f t y organizations which met the f i r s t two c r i t e r i a were l i s t e d i n the two source documents, and from that group of companies, those which were known to be subsidiaries of larger corporations i n the group were withdrawn, as were companies which were known to be conducting no management or supervisory t r a i n i n g . In addition, where any one business category such as banking or lumbering was represented by a large number of companies, a sample of those firms was randomly chosen, and the remainder were withdrawn so that the desired t o t a l of one hundred companies was retained. The panel.included a substantial proportion of the companies employing at least two hundred people i n greater Vancouver, and was therefore considered to be somewhat representative of the o v e r a l l group of larger corporations i n the area. The personnel manager, tra i n i n g director, or another appropriate person within each chosen company was approached f i r s t by l e t t e r and then by telephone with a request for company p a r t i c i p a t i o n i n the survey. A c r i t e r i o n of four telephone c a l l s was established as the l i m i t of attempts to reach the appropriate company spokesman after the introductory l e t t e r s were forwarded. From the o r i g i n a l panel of one hundred companies selected for the study, interviews were completed with f i f t y - n i n e . Of those f i f t y - n i n e firms, twenty-three were ac t i v e l y engaged i n the l o c a l planning and implementation of t r a i n i n g programs for the i r managers and supervisors within B r i t i s h Columbia. Interviews which lasted between f o r t y - f i v e minutes and two hours were conducted with those twenty-three companies, and the interview schedule may be found i n Appendix E. The remaining t h i r t y - s i x companies had developed and planned no such t r a i n i n g programs, although a limited number were beginning to introduce t r a i n i n g systems into t h e i r l o c a l organizations, and others were sending employees to educational i n s t i t u t i o n s or to corporate head o f f i c e s i n eastern Canada for t r a i n i n g programs. Abbreviated interviews were conducted with the t h i r t y - s i x firms, since no information r e l a t i n g to t r a i n i n g programs or t r a i n i n g systems on the l o c a l scene was available. Some of the reasons that the remaining forty-one companies i n the panel were not interviewed included the following: (1) no longer i n business i n Vancouver ( 2 ) were found to be a subsidiary of .another firm i n the panel (3) declined to pa r t i c i p a t e (4) no longer employed two hundred people (5) could not be contacted within four telephone c a l l s (6) a mutually convenient interview time could not be arranged. The d i s p o s i t i o n of the one hundred companies i n the panel may be seen i n Table VII. 71 TABLE VII DISPOSITION OF INTERVIEW REQUESTS FOR COMPANIES IN SURVEY PANEL DISPOSITION NUMBER COMPANIES INTERVIEWED Companies Conducting T r a i n i n g Companies Not Conducting T r a i n i n g 23 36 59 COMPANIES NOT INTERVIEWED No Longer Employ Two Hundred People D e c l i n e d to P a r t i c i p a t e C o u l d Not Contact W i t h i n Four Telephone C a l l s Could Not Arrange Convenient I n t e r v i e w Time S u b s i d i a r y of Another Firm i n the Panel No L o n g e r . i n Business i n Vancouver 41 9 8 7 7 5 5 100 For each o f the twenty-three companies found to be conducting l o c a l t r a i n i n g programs, a number of p i e c e s of i n f o r m a t i o n were c o l l e c t e d r e l a t e d t o the company i n g e n e r a l , to i t s t r a i n i n g system, and to the use of the twenty-six s p e c i f i c b ehaviours i n c l u d e d i n the Index of Program P l a n n i n g . From t h a t i n f o r m a t i o n , estimates o f r e l i a b i l i t y and v a l i d i t y were developed f o r the index. The Panel The twenty-three companies i n which management and s u p e r v i s o r y t r a i n i n g programs were conducted l o c a l l y were d i s t r i b u t e d among a number o f d i f f e r e n t c a t e g o r i e s w i t h r e s p e c t to t h e i r nature of b u s i n e s s . As may be noted i n Table V I I I , f i v e f i r m s were i n the f o r e s t r y b u s i n e s s , four were i n manufacturing and p r o c e s s i n g , one was i n c o n s t r u c t i o n , f i v e were i n t r a n s p o r t a t i o n 72 TABLE VIII NATURE OF BUSINESS FOR COMPANIES CONDUCTING MANAGEMENT TRAINING NATURE OF BUSINESS NUMBER Forestry Manufacturing or Processing Construction Transportation or Communications or U t i l i t i e s R e t a i l or Wholesale Trade Finance 5 4 1 5 6 2 23 or communications or u t i l i t i e s , six were i n the wholesale and r e t a i l trades, and two were i n finance. With respect to ownership, f i v e firms were p r i v a t e l y owned, f i f t e e n were p u b l i c l y owned, and three were crown corporations. The head o f f i c e s of a l l twenty-three companies were located i n Canada. Fourteen of those head o f f i c e s were located i n B r i t i s h Columbia, two were i n the p r a i r i e provinces, three were i n Ontario, and four were i n Quebec. The average number of people employed by the twenty-three companies was 10,932, with a range of from 200 to 50,000. Eight firms employed more than 10,000 people and four employed less than 1,000. Within B r i t i s h Columbia alone, the average number of people employed was 3,900, with a range of from 200 to 18,000. Seven companies employed more than 5,000 people i n B r i t i s h Columbia, and six firms employed less than 1,000. Information r e l a t i n g to the company p a y r o l l was volunteered by fourteen companies, and for that group the average p a y r o l l amounted to $74,000,000 annually, with a range of from $5,000,000 73 to $231,000,000. The annual p a y r o l l s f o r three f i r m s were over $100,000,000 and three p a y r o l l s were l e s s than $20,000,000. Twenty c o r p o r a t i o n s p r o v i d e d i n f o r m a t i o n r e l a t i n g to the gross s a l e s o f goods or s e r v i c e s , and the average value o f s a l e s f o r those companies was $381,000,000, w i t h a range of from $10,000,000 to $1,079,000,000. The s a l e s of four firms were over $800,000,000 and three companies grossed l e s s than $100,000,000. RELIABILITY AND -VALIDITY The q u a l i t y of program p l a n n i n g w i t h i n each of the twenty-three p a r t i c i p a t i n g companies was r e p r e s e n t e d by a t o t a l score on the Index o f Program Planning. That score was d e r i v e d by a p p l y i n g the i n d i v i d u a l weights o f importance f o r each o f the twenty-six items w i t h i n the index to the r e p o r t e d per cent o f use w i t h r e s p e c t to the a p p r o p r i a t e c o r p o r a t e p l a n n i n g procedures, and adding the twenty-six products t o g e t h e r . The g r e a t e s t score p o s s i b l e was 505.3, and the average score a t t a i n e d by. the twenty-three c o r p o r a t i o n s was 293.1, which was approximately f i f t y - e i g h t per cent of the p o s s i b l e t o t a l . The scores ranged from 142.7 to 436.3, wi t h three scores below 2 00 and four scores above 350. R e l i a b i l i t y An estimate of i n t e r n a l r e l i a b i l i t y was d e r i v e d f o r the Index o f Program Planning by c a l c u l a t i n g a c o e f f i c i e n t o f i n t e r n a l c o n s i s t e n c y , u s i n g an a n a l y s i s of v a r i a n c e procedure o u t l i n e d by Winer. (57:124) Greater i n t e r n a l r e l i a b i l i t y f o r the index was a s s o c i a t e d w i t h h i g h e r p o s i t i v e c o r r e l a t i o n s . The a n a l y s i s of v a r i a n c e procedure i n c o r p o r a t e d as b a s i c data, the twenty-six item products l i s t e d f o r each o f the twenty-three companies, and a standard one-way a n a l y s i s o f v a r i a n c e was conducted to 74 obtain the between-companies and within-companies variances. The within-companies variance was then divided into the portions which were due to between-items variance and to r e s i d u a l error. For purposes of the r e l i a b i l i t y formula, less r e s i d u a l error was associated with greater variance accounted for by the in d i v i d u a l item differences, and hence, with greater r e l i a b i l i t y for the index. Lesser r e l i a b i l i t y was associated with greater r e s i d u a l error, or variance not accounted for by the differences between items. The formula for the c a l c u l a t i o n of r e l i a b i l i t y , and the appropriate analysis of variance data may be seen i n Table IX. As may be noted, the c o e f f i c i e n t of i n t e r n a l consistency for the Index of Program Planning was +0.85. TABLE IX COEFFICIENT OF INTERNAL CONSISTENCY FOR THE INDEX OF PROGRAM PLANNING SOURCE OF VARIANCE ANALYSIS OF VARIANCE D.F. SS MS Between Companies Within Companies Between Items Residual Error Total 22 575 25 550 597 4,823.70 43,878.75 25,455.76 18,422.99 48, 702.45 219.26 33.50 _ , . , MSn . n - MS„ 219.26 - 33.50 R e l i a b i l i t y = Bet .Com. Res. = = +0.85 MS Bet.Com. 219.26 75 V a l i d i t y In assessing the Index of Program Planning, three types of v a l i d i t y were considered. Face v a l i d i t y was assumed to be s a t i s f a c t o r y as a r e s u l t of the opinions of the judges with respect to the importance of i n d i v i d u a l program planning behaviours. Any behaviour which was consistently rated high by judges was assumed to be a v a l i d measure of better q u a l i t y program planning, and s i m i l a r l y , any item rated consistently low was assumed to v a l i d l y represent a lower qu a l i t y of program planning. A c o l l e c t i o n of such items, weighted with respect to importance, appeared to be v a l i d as an o v e r a l l measure of qu a l i t y for a set of program planning procedures, and the index was therefore considered to have s a t i s f i e d the conditions for face v a l i d i t y . The requirements for content v a l i d i t y were also assumed to have been met by the Index of Program Planning. A representative sample of program planning items was included i n the index as a r e s u l t of two steps i n the item selection procedures: (1) the development of a seven-category program planning model which represented a l l aspects of program planning procedures f e l t to be important i n business and industry (2) a sequence of judgements from knowledgeable people with respect to the v a l i d i t y and appropriateness of i n d i v i d u a l items for the index. While no s p e c i f i c tests were conducted to assess either face v a l i d i t y or content v a l i d i t y , the assessment of construct v a l i d i t y required further analysis. There appeared to be no existing measures of q u a l i t y for program planning procedures i n business and industry, and as a re s u l t , the construct v a l i d i t y of the index could not be established through c o r r e l a t i o n with existing constructs or known observables which were assumed to be associated with the qu a l i t y of program planning. In the absence of established measures of program planning quality or of other t h e o r e t i c a l constructs, three hypotheses were developed as a representation of a l o g i c a l framework by which the construct v a l i d i t y of the index might be indicated. The estimation of construct v a l i d i t y was considered to be only generally i n d i c a t i v e i n format, rather than s p e c i f i c a l l y established with reference to fa c t . Each of the three hypotheses attempted to associate the q u a l i t y of program planning, as represented by the Index of Program Planning score, with an independent variable for which several d i f f e r e n t measures were possible. Since the hypotheses were developed i n an exploratory format, and the test r e s u l t s were meant to be broadly interpreted, a si g n i f i c a n c e l e v e l of .10 was chosen for the hypothesis testing procedures. Where the data for independent variables were of an i n t e r v a l or r a t i o form, regression techniques were used i n the s t a t i s t i c a l analysis, and where the data were of a nominal form, analysis of variance procedures were used. Hypothesis One Better qu a l i t y program planning w i l l be found among corporations with greater economic power and influence. The r e l a t i o n s h i p between the qu a l i t y of program planning and the economic power and influence of corporations was tested i n consideration of the f i r s t hypothesis. That hypothesis was established upon the l o g i c a l , though untested, assumption that larger and more powerful companies would more l i k e l y have greater resources for the provision of more highly q u a l i f i e d 77 program planning s t a f f . It was also assumed that larger organizations might require more sophisticated planning procedures so that t r a i n i n g programs might be implemented within greater degrees of organizational complexity. Five d i f f e r e n t measures of economic power and influence were tested with respect to their association with quality of program planning, and as may be noted i n Table X, none of the f i v e were found to be s t a t i s t i c a l l y s i g n i f i c a n t . The quality of program planning was not s i g n i f i c a n t l y associated with the t o t a l number of people employed by the company i n Canada, the number of people employed by the company i n B r i t i s h Columbia, the amount of the company pay r o l l , or the annual value of gross sales for goods and services. Furthermore, no s i g n i f i c a n t association was found between qua l i t y of program planning and the p r o v i n c i a l location of company head o f f i c e , where the head o f f i c e s were located i n B r i t i s h Columbia, Ontario, Quebec, or the p r a i r i e provinces. TABLE X STATISTICAL ANALYSES FOR HYPOTHESIS ONE MEASURE OF ECONOMIC POWER AND INFLUENCE F DF R N SIG. Total Number of Employees Number of Employees i n B r i t i s h Columbia Annual Company Pa y r o l l Annual Gross Sales Location of Head O f f i c e 1.17 3,19 .16 .25 .03 .09 23 23 14 20 N.S. N. S. N.S. N.S. N.S. Hypothesis Two Better qu a l i t y program planning w i l l be found among corporations which exhibit greater commitment toward the tr a i n i n g and personnel development function. 78 The focus of the second hypothesis was the r e l a t i o n s h i p between qual i t y of program planning and commitment toward tr a i n i n g and personnel development. Although no research findings were available, commitment toward t r a i n i n g was often c i t e d i n the l i t e r a t u r e as an important element i n the success of t r a i n i n g operations within companies. Ten d i f f e r e n t measures of commitment were included i n the testing series for the second hypothesis. As may be noted i n Table XI, no s i g n i f i c a n t relationships were found between qual i t y of program planning and the existence of a written t r a i n i n g p o l i c y within the company; the r e l a t i v e p o s i t i o n of the person d i r e c t l y responsible for training, as upper, middle, or lower management i n the organization; the number of years i n which management and supervisory t r a i n i n g programs had been continuously offered i n the company; or the number of days per year normally allocated by the company for the attendance of supervisors at company tr a i n i n g programs or educational programs offered by other agencies. With respect to program planning quality and a number of other measures of commitment, however, s t a t i s t i c a l l y s i g n i f i c a n t relationships were found. The r e l a t i o n s h i p between qual i t y of program planning and the per cent of management and supervisory t r a i n i n g which was conducted during company time was found to be positive, and s i g n i f i c a n t at the .08 l e v e l . Companies which conducted more of their management and supervisory t r a i n i n g on company time, rather than at night or on weekends, appeared to exhibit higher qu a l i t y i n their program planning procedures. The r e l a t i o n s h i p between qual i t y of program planning and the per cent of managers and supervisors who had participate d i n company t r a i n i n g programs within twenty-four months was also 79 TABLE XI STATISTICAL ANALYSES FOR HYPOTHESIS TWO MEASURE OF COMMITMENT TOWARD TRAINING F DF R N SIG. Per Cent o f T r a i n i n g on Company Time .36 23 .08 Years o f Continuous T r a i n i n g Programs .05 21 N.S. Degree of P a r t i c i p a t i o n i n T r a i n i n g .41 22 .06 Days A l l o c a t e d f o r Manager T r a i n i n g .43 21 .05 Days A l l o c a t e d f o r Supervisor T r a i n i n g . -31 21 N.S. Reimbursement f o r E d u c a t i o n a l Courses -.43 23 .04 E x i s t e n c e o f W r i t t e n T r a i n i n g P o l i c y 2 .47 1,21 N.S. W i l l i n g n e s s to Spend f o r T r a i n i n g 4 .68 1,21 .04 Management A t t i t u d e Toward T r a i n i n g 4 .34 2,20 .03 Management L e v e l o f T r a i n i n g D i r e c t o r 2 .02 2,20 N. S. found to be s i g n i f i c a n t . The r e l a t i o n s h i p was p o s i t i v e and s i g n i f i c a n t at the .06 l e v e l , i n d i c a t i n g t h a t b e t t e r q u a l i t y program p l a n n i n g was e x h i b i t e d i n companies where a g r e a t e r per cent of the management and s u p e r v i s o r y s t a f f had been i n v o l v e d i n company t r a i n i n g programs over the p r e v i o u s twenty-four months. A t h i r d r e l a t i o n s h i p which was found to be s i g n i f i c a n t concerned the days per year which were a l l o c a t e d by companies f o r the attendance of managers at company t r a i n i n g programs or e d u c a t i o n a l programs sponsored by other agencies. The r e l a t i o n s h i p was p o s i t i v e and s i g n i f i c a n t at the .05 l e v e l , i n d i c a t i n g t h a t b e t t e r q u a l i t y program p l a n n i n g was e x h i b i t e d by companies which a l l o c a t e d a g r e a t e r number o f days per year f o r the t r a i n i n g of managers w i t h i n e d u c a t i o n a l events o f a formal nature. The degree to which companies reimbursed t h e i r employees f o r c o s t s i n c u r r e d i n t a k i n g e d u c a t i o n a l courses sponsored by agencies other than the company was found to be s i g n i f i c a n t l y 80 related to the qua l i t y of program planning. The relationship, however, was negative with a c o r r e l a t i o n of -.43 which was s i g n i f i c a n t at the .04 l e v e l . Companies which reimbursed employees for educational courses to a greater extent exhibited a lower q u a l i t y of program planning, perhaps indicating that external educational courses may have been used as a substitute for i n t e r n a l t r a i n i n g programs i n some instances. The qua l i t y of program planning was found to be s i g n i f i c a n t l y associated with the willingness of the company to set aside funds for worthwhile company tra i n i n g programs. Those companies which were very w i l l i n g to spend money for t r a i n i n g exhibited better q u a l i t y program planning than those companies which were only moderately w i l l i n g to spend money for t r a i n i n g . The r e l a t i o n s h i p was s t a t i s t i c a l l y s i g n i f i c a n t at the .04 l e v e l , as indicated by an F r a t i o of 4.68 with 1 and 21 degrees of freedom. The attitude of top management toward company tr a i n i n g programs was also s i g n i f i c a n t l y associated with the q u a l i t y of program planning. Those companies i n which the attitudes of top management toward tr a i n i n g were generally very favorable, rather than somewhat favorable or non-commital, exhibited better q u a l i t y program planning. That r e l a t i o n s h i p was indicated by an F r a t i o of 4.34, with 2 and 2 0 degrees of freedom, which was s i g n i f i c a n t at the .03 l e v e l . Hypothesis Three Better qu a l i t y program planning w i l l be found among corporations with more highly developed t r a i n i n g and personnel development systems. Hypothesis three was also based upon a l o g i c a l framework rather than a t h e o r e t i c a l or research-based foundation. However, 81 the development of the t h i r d hypothesis was an extension of Shaffer's research findings that evaluation e f f o r t s for management development were greater i n companies with larger management tr a i n i n g s t a f f s and where the tr a i n i n g department had been i n operation longer. (48) The focus of hypothesis three was the r e l a t i o n s h i p between qua l i t y of program planning and the state of development within the corporate t r a i n i n g system. Some of the measures which attempted to indicate the sophistication of the tr a i n i n g department and the competence of i t s s t a f f included the number of tra i n i n g s p e c i a l i s t s f u l l y employed on the t r a i n i n g staff,- the formal t r a i n i n g attained i n educational f i e l d s by those trainers, the per cent of trainers who had been transferred to the tr a i n i n g department from another company department, the years of experience i n corporate t r a i n i n g among the tr a i n i n g s t a f f , the number of programs offered by the tr a i n i n g department over twenty-four months, the amount of budget allocated for training, the per cent of budget allocated to the functions of instruction, materials, evaluation, and planning, and the subscription by the company to either of the magazines Training i n Business and Industry or Training and Development  Journal. As may be noted i n Table XII, the r e l a t i o n s h i p between the majority of those measures and q u a l i t y of program planning was found to be not s i g n i f i c a n t . Of the two relationships which were found to be s t a t i s t i c a l l y s i g n i f i c a n t , the f i r s t concerned the per cent of t r a i n i n g s t a f f who had received some formal tr a i n i n g i n educational subject matter. Better qu a l i t y program planning was exhibited by companies i n which a higher per cent of the tr a i n i n g s t a f f had received t r a i n i n g i n an educational f i e l d , 82 TABLE XII STATISTICAL ANALYSES FOR HYPOTHESIS THREE MEASURE OF DEVELOPMENT IN TRAINING F DF R N SIG. Number of Trainers Employed .10 23 N. S. Educational Training for Trainers .47 16 .06 Trainers Transferred From Other Department .07 16 N. S. Fewer Years Experience i n Training .62 16 .009 Number of Programs Offered .28 22 N.S. Amount of Training Budget -.04 15 N.S. Per Cent of Budget for Instruction .27 18 N. S. Per Cent of Budget for Materials -.20 16 N.S. Per Cent of Budget for Evaluation .20 16 N.S. Per Cent of Budget for Planning .11 18 N.S. Subscription to Training Magazines 1.62 1,21 N.S. and that r e l a t i o n s h i p was indicated by a c o r r e l a t i o n of .47 which was s i g n i f i c a n t at the .06 l e v e l . The other r e l a t i o n s h i p which was found to be s i g n i f i c a n t focused upon the number of years of experience i n management trai n i n g which was reported by the t r a i n i n g s t a f f . In a negative relationship, better q u a l i t y program planning was exhibited by companies i n which a lesser per cent of the tr a i n i n g s t a f f had been involved i n the management tr a i n i n g area for less than two years. The c o r r e l a t i o n of -.62 was s i g n i f i c a n t at the .009 le v e l , and indicated that better q u a l i t y program planning was evident among t r a i n i n g s t a f f s with greater experience i n the management trai n i n g area. Interpretation of the Hypotheses On the basis of the three hypotheses and the variety of independent measures associated with them, i t would appear that 83 a l o g i c a l case might be outlined to j u s t i f y the construct v a l i d i t y of the Index of Program Planning, despite a number of variables which were found to be not s i g n i f i c a n t and contrary to expectations. A series of variables r e l a t i n g to the size and economic power of both the company and i t s t r a i n i n g operations was found to be not s i g n i f i c a n t . Those variables contributed to the r e j e c t i o n of the f i r s t and t h i r d hypotheses, and indicated that better quality program planning was not associated with larger organizations or more extensive t r a i n i n g systems. It appeared that greater provision of s t a f f and funds was not s u f f i c i e n t to ensure better q u a l i t y program planning. However, a consistent pattern was evident among those remaining relationships which were found to be s t a t i s t i c a l l y s i g n i f i c a n t . Better q u a l i t y program planning was exhibited by those companies which appeared to place some degree of emphasis and commitment on their own i n t e r n a l management programs, and less emphasis on external programs offered by other educational agencies. A better q u a l i t y of program planning was found to be associated with a greater per cent of tr a i n i n g conducted on company time, a greater per cent of management p a r t i c i p a t i o n i n the programs, greater numbers of company days allocated for management tra i n i n g purposes, greater willingness to spend money for worthwhile t r a i n i n g programs, more favorable attitudes toward t r a i n i n g on the part of top management, and lesser reimbursement for external educational programs. With those conditions setting a favorable environment for the conduct of management and supervisory training programs, i t might l o g i c a l l y be concluded that company tra i n i n g departments staffed to a greater extent 84 by trainers with formal t r a i n i n g i n educational f i e l d s and with more years of experience i n the company training system, would more l i k e l y be i n a p o s i t i o n to provide better qu a l i t y program planning services, as was found to be true i n the s t a t i s t i c a l analyses. Therefore, i n consideration of the l o g i c a l framework and the associated testing procedures which were established for purposes of assessing the Index of Program Planning, s u f f i c i e n t evidence appears to have been accumulated to suggest that the hypothetical construct for quality i n program planning was v a l i d l y measured by the Index of Program Planning. CHAPTER FIVE SUMMARY AND CONCLUSIONS This research project was undertaken as part of the small, but rapidly growing research f i e l d which i s attempting to develop more adequate personnel development procedures i n business and industry. Over the past three decades, i n which formal management development has become an accepted a c t i v i t y within the business community, a series of guidelines r e l a t i n g to successful tr a i n i n g practices has grown within the ranks of professional trainers, but those guidelines have generally not been systematically tested nor widely disseminated. Even with the existence of such guidelines, tr a i n i n g programs i n industry have often been i n e f f e c t i v e or unsuccessful, and a number of reasons associated with program f a i l u r e s have been suggested. In most instances those suggested reasons were related i n some manner to the program planning procedures which were used i n the development of programs. The presentation of formal programs has been both an important and highly v i s i b l e function among corporate tr a i n i n g a c t i v i t i e s , and therefore, attempts to improve personnel development practices i n companies have often related to possible improvements i n the qu a l i t y of such programs. L i t t l e research has been conducted with respect to program planning procedures for tr a i n i n g purposes, and with many business and i n d u s t r i a l concerns seeking improvements i n their t r a i n i n g practices and coping with a rapi d l y expanding emphasis on personnel development, th i s research study attempted to focus upon the program planning area of concern. 85 86 SUMMARY The major purpose of the study was to develop an Index of Program Planning which might r e l i a b l y and v a l i d l y measure the standard of qu a l i t y present among a set of program planning procedures o r d i n a r i l y exhibited by a company i n the development of i t s managerial and supervisory t r a i n i n g programs. An instrument with the c a p a b i l i t y for objectively measuring standards of q u a l i t y i n program planning was f e l t to be useful i n a number of p r a c t i c a l and t h e o r e t i c a l s i t u a t i o n s . For example, i n d i v i d u a l companies might use such an index to i d e n t i f y areas of planning weakness and to guide possible improvements i n planning procedures, while researchers might use an index to es t a b l i s h norms of planning quality, to test associations between qu a l i t y i n program planning and a variety of organizational factors, or to analyze relationships between program effectiveness and qu a l i t y i n planning. The study was conducted i n a sequential manner, commencing with the development of a conceptual model for program planning i n business and industry, followed by several developmental phases r e l a t i n g to the Index of Program Planning, and concluding with a survey of companies for the purpose of testing the index with respect to r e l i a b i l i t y and v a l i d i t y . In an attempt to integrate some of the p r a c t i c a l guidelines which have been suggested for conducting i n d u s t r i a l training, and some of the written materials r e l a t i n g to program planning both i n business and i n other areas of interest, a conceptual model for t r a i n i n g i n business and industry was developed. That model focused on the program planning process, and i t consisted of seven major planning functions: (1) corporate analysis and 87 manpower planning (2) determining and assessing t r a i n i n g needs (3) designing the t r a i n i n g program (4) l e g i t i m i z a t i o n (5) i n s t r u c t i o n a l support (6) evaluation (7) maintenance of behaviour. The l i t e r a t u r e r e l a t i n g to t r a i n i n g was then searched for i l l u s t r a t i o n s of t y p i c a l corporate behaviours, and for each of the seven major functions, eight statements were constructed to specify behaviours which might be performed by a company i n the conduct of that function. A series of structured exercises based upon the model and i t s f i f t y - s i x statements was then designed. The exercises required the p a r t i c i p a t i o n of people who were knowledgeable with respect to either management t r a i n i n g or to theory i n program planning. Sixty-nine people, most of whom resided i n western Canada, participated i n the study as judges, and provided their opinions i n accordance with the requirements of the structured exercises. The f i r s t exercise was designed to eliminate those statements which were not considered to be v a l i d examples of the planning function which they had been intended to represent. Only those statements which obtained a median score of 3.2 5 or more with respect to a five-point scale of v a l i d i t y were retained for further consideration as statements to be included i n the Index of Program Planning. A group of eighteen judges completed exercise one, and as a r e s u l t of their opinions, f i v e of the o r i g i n a l f i f t y - s i x statements were withdrawn from further consideration. Exercise two was designed to eliminate from the l i s t of f i f t y - s i x statements, those representing behaviours for which there was no agreement among judges with respect to the 88 appropriate function i n the planning process. Each of sixteen judges, none of whom had participated i n exercise one, was independently provided with a random l i s t of the f i f t y - s i x statements, and was asked to categorize each statement into i t s appropriate planning function. Six statements did not meet the c r i t e r i o n l e v e l of sixty per cent correct categorization, and were eliminated from further consideration as possible statements i n the Index of Program Planning. With f o r t y - f i v e of the o r i g i n a l statements remaining for consideration after the completion of exercises one and two, the th i r d and f i n a l exercise was designed. The exercise consisted of the f o r t y - f i v e statements ordered with respect to the seven major planning functions. Judges were asked to rate the importance of each statement i n planning and implementing a management or supervisory t r a i n i n g program, and were provided with a nine-point scale of importance to use for that purpose. The objective of the t h i r d exercise was to select .a number of statements for the Index of Program Planning on the basis of agreement among judges with respect to the importance of behaviours i n planning management tra i n i n g programs. The exercise was completed by f i f t y - e i g h t judges. The data derived from exercise three were i n an inappropriate form for further s t a t i s t i c a l analysis, because of the unequal and unknown size of the intervals on the nine-point scale of importance. The f o r t y - f i v e o r i g i n a l d i s t r i b u t i o n s were therefore transformed onto a common underlying continuum constructed using the method of successive i n t e r v a l s . The semi-interquartile range for each d i s t r i b u t i o n on the common continuum was used as a measure of agreement, and those statements for which that s t a t i s t i c was more than the 1.36 standard within the unit-normal curve 89 were not included i n the Index of Program Planning. Nineteen statements were withdrawn as a r e s u l t of the greater-than-normal disagreement among judges i n exercise three with respect to the importance of stated behaviours i n the program planning process. The Index of Program Planning consisted of those twenty-six statements from the o r i g i n a l f i f t y - s i x which met the c r i t e r i a for each of the three structured exercises. Therefore, each statement which was included i n the index exhibited the following c h a r a c t e r i s t i c s : (1) the statement was a v a l i d example of the planning function i t represented (2) the major planning function of the statement i n the program planning process was generally agreed upon by a panel of knowledgeable judges (3) the importance of the statement i n the program planning process was generally agreed upon by a panel of knowledgeable judges. The index, as a compilation of i n d i v i d u a l statements representing various levels of importance i n the program planning process, was designed i n consideration of three assumptions: (1) better q u a l i t y program planning would be indicated by the use of more of the seven major functions i n the planning model (2) better q u a l i t y program planning would be indicated by the use of a greater number of d i f f e r e n t behaviours i n the conduct of any p a r t i c u l a r planning function (3) better q u a l i t y program planning would be indicated by the use of more important behaviours rather than less important behaviours i n the conduct of any planning function. The Index of Program Planning was designed to provide a numerical value which would represent a cumulative i n d i c a t i o n of the standard of quality exhibited by a company i n i t s program planning procedures. A l l numerical calculations i n the index were based upon the values of importance which were associated 90 with the twenty-six statements retained for the instrument- The values of importance were derived by locating the median value of importance for the d i s t r i b u t i o n on the common underlying continuum i n association with exercise three, and multiplying that value by a factor of ten. A t o t a l score was determined for any company, using the index, by combining the value of importance and the r e l a t i v e per cent of use for each of the twenty-six behaviours l i s t e d i n the index, and summing the twenty-six i n d i v i d u a l products from those m u l t i p l i c a t i o n s . The highest score possible on the index was 505.3. The score for any company using the index to measure i t s qua l i t y of program planning procedures would vary with the s p e c i f i c behaviours generally used, but would be greater i n d i r e c t proportion to a combination of the greater number of d i f f e r e n t behaviours used, to the greater importance of those behaviours, and to the more extensive use of each of the behaviours over a period of time. For the purpose of estimating the r e l i a b i l i t y and v a l i d i t y of the Index of Program Planning as a measurement instrument, a survey of large corporations was undertaken. A panel of one-hundred companies was selected from l i s t s of companies within appropriate publications, according to the following c r i t e r i a : (1) the company was engaged i n business within the greater Vancouver area of B r i t i s h Columbia (2) the company was l i s t e d as employing two hundred people or more (3) the company was not a subsidiary of another firm i n the panel. Interviews were sought with each of the firms, and f i f t y - n i n e interviews were conducted. Twenty-three of the companies interviewed conducted their own l o c a l management and supervisory training, and for the interview schedules which were used with those firms, the Index of Program Planning was included. 91 With respect to the type of business engaged i n by the twenty-three companies which were surveyed, six were i n the r e t a i l or wholesale trades, fiv e were i n forestry, f i v e were i n transportation or communications or u t i l i t i e s , four were i n manufacturing or processing, two were i n finance, and one was i n construction. Five companies were pr i v a t e l y owned, f i f t e e n were p u b l i c l y owned, and three were crown corporations. A l l twenty-three head o f f i c e s were i n Canada, and fourteen were located i n B r i t i s h Columbia. The average number of people employed by the companies was 10,932, with an average of 3,900 people employed i n B r i t i s h Columbia. The average annual p a y r o l l for the fourteen companies who volunteered the information was $74,000,000, and the average gross sales of twenty firms was $381,000,000. Within the twenty-three corporations which were used to assess the Index of Program Planning, the highest score attained on the index was 436.3 from the possible t o t a l of 505.3. The average score attained was 291.3, which was f i f t y - e i g h t per cent of the possible t o t a l . The r e l i a b i l i t y for the Index of Program Planning was estimated using a one-way analysis of variance procedure which incorporated as i t s data the twenty-six i n d i v i d u a l item products and the t o t a l index score for each of the twenty-three p a r t i c i p a t i n g corporations. The estimated c o e f f i c i e n t of r e l i a b i l i t y was +0.85. The construct v a l i d i t y for the Index of Program Planning was estimated through the use of three hypotheses. The hypotheses were developed as a representation of a l o g i c a l framework, rather than as a r e s u l t of previously established theory or research r e s u l t s . Each of the hypotheses consisted of the major dependent 92 variable, which was the quality of program planning, and a conceptual independent variable for which several d i f f e r e n t measures were possible. The hypotheses were tested with analysis of variance and regression techniques, using the .10 l e v e l of si g n i f i c a n c e . Hypothesis one, that better qu a l i t y program planning would be found among corporations with greater economic power and influence, was tested i n r e l a t i o n to five d i f f e r e n t measures, and none was' found to be s i g n i f i c a n t . No s i g n i f i c a n t r e lationships were found between q u a l i t y of program planning and the t o t a l number of people employed i n the company, the number of people employed i n B r i t i s h Columbia, the amount of the annual p a y r o l l , the amount of annual gross sales, or the p r o v i n c i a l location of corporate head o f f i c e . Hypothesis two, that better qu a l i t y program planning would be found among corporations which exhibited greater commitment toward t r a i n i n g and personnel development, was tested i n r e l a t i o n to ten d i f f e r e n t measures. Six of those measures were found to be s i g n i f i c a n t l y associated with the qua l i t y of program planning. However, no s i g n i f i c a n t relationships were found between qua l i t y of program planning and four measures, which included the number of years for which tr a i n i n g programs had been continuously offered by the company, the number of days allocated by the company for supervisors to attend t r a i n i n g programs, the existence of a written t r a i n i n g policy, or the management l e v e l i n the organization which was held by the director of t r a i n i n g . Five measures which were found to be s i g n i f i c a n t l y associated i n a posi t i v e d i r e c t i o n with better qu a l i t y program planning included the per cent of t r a i n i n g conducted on company time, 93 the per cent of managers and supervisors who had participated i n company tra i n i n g programs within the previous twenty-four months, the number of days allocated by the company for managers to attend t r a i n i n g programs, the willingness of the company to spend money for worthwhile tr a i n i n g programs, and the attitude of top management toward the tr a i n i n g function. One other measure, the per cent of company reimbursement for employees enrolled i n educational courses sponsored by external agencies, was s i g n i f i c a n t l y associated with better qu a l i t y program planning in a negative d i r e c t i o n . Hypothesis three, that better quality program planning would be found among corporations with more highly developed t r a i n i n g and personnel development systems, was tested i n r e l a t i o n to eleven d i f f e r e n t measures. Of those measures, nine were found to be not s i g n i f i c a n t , and they included the number of t r a i n i n g s p e c i a l i s t s employed by the company i n B r i t i s h Columbia, the per cent of tra i n i n g s t a f f who had been transferred from other company departments, the number of d i f f e r e n t t r a i n i n g programs affered for management and supervisory s t a f f , the amount of the annual t r a i n i n g budget, the per cent of the budget allocated for i n s t r u c t i o n or materials or evaluation or program planning, or the subscription by the company to professional t r a i n i n g journals. With respect to the two remaining measures, better quality program planning was found to be s i g n i f i c a n t l y associated with the greater per cent of tr a i n i n g s t a f f who had received some formal t r a i n i n g i n an educational f i e l d , and a lesser per cent of tr a i n i n g s t a f f who had less than two years experience i n the corporate t r a i n i n g function. 94 Because of the l o g i c a l consistency of the set of s i g n i f i c a n t associations which were found i n the hypotheses, indicating that better quality program planning was more evident among companies which placed some emphasis and commitment on their own i n t e r n a l management t r a i n i n g programs, i t was concluded that a claim of construct v a l i d i t y for the Index of Program Planning was j u s t i f i e d . CONCLUSIONS The conclusions which were drawn as a r e s u l t of t h i s study focused on two major areas of concern: (1) the methodological approach used i n the study (2) the Index of Program Planning. In general, i t may be concluded that the methodology which was used for t h i s study was successful i n achieving the desired objectives, and that similar methods might be used i n the construction of measurement instruments for program planning i n other areas of a c t i v i t y . The methodological approach, which included the conceptualization of a planning model, selection of statements to i l l u s t r a t e appropriate behaviours with respect to that model, and sequential testing of those statements with respect to a variety of c r i t e r i a , might be applied to the specialized sequences of program planning for a variety of inter e s t areas within adult education. Although the o v e r a l l approach was found to be satisfactory, suggested improvements for i t s further use i n other studies might re l a t e to such aspects as the selection of judges on a more c a r e f u l l y controlled basis to achieve greater representation from selected organizations or geographical areas, the addition of further procedures for item selection to achieve proportional balances among the number of statements representing various 95 segments of the planning model i n the f i n a l measurement instrument, or the incorporation of a Delphi technique into the judgement procedures so that greater consensus might be achieved through repeated cycles of judgements and information feedback with the group of judges. As examples of those possible improvements, the present study might have selected the group of judges more c a r e f u l l y with respect to their organizational a f f i l i a t i o n or geographical location so as to achieve some desired balance of demographic c h a r a c t e r i s t i c s within either the t o t a l group or any of the sub-groups which were used for selected exercises. Additional steps might also have been taken to ensure that a pre-determined number of statements were selected for the index from each of the seven planning functions to represent either an equal balance or some other proportional representation among the functions. The study might also have incorporated a Delphi technique within the judgements of importance i n an attempt to reduce the lack of consensus among the opinions of judges, and i n that manner, to develop a more c a r e f u l l y considered set of judgements upon which to select items. The major conclusion to be drawn from t h i s study was that the Index of Program Planning was a v a l i d and r e l i a b l e measurement instrument with respect to the purposes for which i t was developed. The analysis of variance test for r e l i a b i l i t y indicated that the index was i n t e r n a l l y consistent, and the hypothesis testing procedures indicated that the index was very l i k e l y a v a l i d measure of the degree of qu a l i t y i n program planning for management and supervisory t r a i n i n g programs i n business and industry i n western Canada. As a r e s u l t of the tests which were conducted with respect to the i n d i v i d u a l measures associated with each of the three 96 hypotheses, a number of secondary conclusions were also derived. Although the tests indicated the existence of s i g n i f i c a n t correlations between variables, they did not infer any necessary causation between those associated variables. Therefore, i t was not possible to conclude that better program planning was, or might be, developed through conscious attempts within corporations to change the status for any of the s i g n i f i c a n t measures which were i d e n t i f i e d i n this study. Nor was i t possible to conclude that the relationships between measures within any single company were i d e n t i c a l with the o v e r a l l r e s u l t s derived from the t o t a l analysis. However, i t was generally concluded that companies which attempted to place more emphasis and importance on their own i n t e r n a l management tr a i n i n g programs through the provision of more adequate support and commitment to the programs were able to exhibit better quality i n program planning. In addition, i t was concluded that better q u a l i t y program planning was exhibited within t r a i n i n g s t a f f s where a greater per cent of the personnel had received formal t r a i n i n g i n educational subject areas, and where a greater per cent of the personnel had more than two years of experience i n the corporate t r a i n i n g function. In l i g h t of those findings, i t might be suggested that companies hoping to improve their program planning procedures might investigate methods of e f f e c t i n g change should a cycle of poor planning and lack of commitment toward programs be found to e x i s t . A further implication might suggest that the q u a l i t y of planning within companies may benefit through the provision of opportunities for the attendance of t r a i n i n g personnel at i n s t r u c t i o n a l events focusing upon educational subject matter, and through the development or a c q u i s i t i o n of more experienced professional t r a i n i n g personnel. 97 In addition to those implications a r i s i n g out of the hypothesis testing r e s u l t s , companies might wish to consider the use of the Index of Program Planning as an a n a l y t i c a l t o o l for their program planning procedures. The index might be used both as an o v e r a l l measure for standards of qual i t y i n planning at one or many points i n time, and as a group of seven sub-scores for purposes of comparing weaknesses and strengths of qual i t y within the d i f f e r e n t functional areas of planning. In either instance, the index may be found to have some p r a c t i c a l value in promoting and guiding possible improvements i n the qual i t y of program planning for corporations. With respect to more t h e o r e t i c a l concerns, the index may be useful to researchers working in the area of t r a i n i n g i n business and industry. Norms of planning qu a l i t y might be established for any number of d i s t i n c t corporate or i n d u s t r i a l groupings, or relationships might be established between the qua l i t y of program planning and a variety of factors present within organizational structures, thus furthering the t h e o r e t i c a l foundations which w i l l be required for the development of more sophisticated t r a i n i n g procedures i n the future. Further work might also be attempted i n exploring the relationships between the success of programs and the qual i t y of planning associated with their development. In that manner, many of the reasons c i t e d for program f a i l u r e s might be more c l o s e l y examined within the framework provided by the index for qual i t y of program planning. At a time when there i s a lack of research tools with respect to personnel development i n business and industry, and when the need for guidance and assistance i n the search for more adequate corporate t r a i n i n g procedures i s apparent, i t may be concluded that the present study has contributed to both p r a c t i c a l and t h e o r e t i c a l areas of i n t e r e s t . 98 BIBLIOGRAPHY Annett, J. "A Systems Approach" i n E. Evans et a l , Planning I n d u s t r i a l Training National Institute of Adult Education (England and Wales), 1968. Bare, I.L. "Training Program Development Processes" Journal of the American Society of Training Directors 12 (3):24-28, 1958. Barrett, J.E. "Can We Evaluate Training Expenditures?" Training i n Business and Industry 7(1):40-55, 1970. Beal, G.M., et a l , Social Action and Interaction i n Program  Planning Ames, Iowa: Iowa State University Press, 1966. Black, J.M. Developing Competent Subordinates New York: American Management Association, 1961. Boone, E.J., Dolan, R.J., and Shearon, R.W. Programming  i n the Cooperative Extension Service: A Conceptual  Schema Raleigh, N.C., The North Carolina A g r i c u l t u r a l Extension Service, North Carolina State University, 1971. Boyle, P.G. The Program Planning Process with Emphasis on  Extension Madison, Wisconson: National A g r i c u l t u r a l Extension Center for Advanced Study, University of Wisconson, 1965. "Planning with P r i n c i p l e s " i n E.J. Boone, et a l , Administration of Adult Education Washington, D.C: Adult Education Association of the U.S.A., 1965. Brandt, S.C. "A No-Nonsense Approach to Employee Training" Business Management 35(4):53-55, 1969. B r i t i s h Columbia Trade Directory, 1971 Government of the Province of B r i t i s h Columbia, Department of I n d u s t r i a l Development, Trade, and Commerce, 1971. Brunner, E. deS., et a l , An Overview of Adult Education  Research Chicago: Adult Education Association of the U.S.A., 1959. Bumstead, R.A. "AT&T Systems Approach for Love and Money" Training i n Business and Industry 5(5):43-46, 1968. 99 Byers, K.T. "Developing E f f e c t i v e Employees and Organizations' i n K.T. Byers (Ed.) Employee Training and Development  i n the Public Service Chicago: Public Personnel Association, 1970. Campbell, J.P. "Personnel Training and Development" Annual Review of Psychology 22:565-602, 1971. Canada Manpower Centre - Area P r o f i l e Vancouver Metropolitan  Area Department of Manpower and Immigration, Manpower Information and Analysis Branch, 1968. Catalanello, R.F. and Kirkpatrick, D.L. "Evaluating Training Programs - the State of the Art" Training  and Development Journal 22(5):2-9, 1968. Coleman, B.P. "An Integrated System for Manpower Planning" Business Horizons 13(5):89-95, 1970. Cone, W.F. "Guidelines for Training S p e c i a l i s t s " Training  and Development Journal 28(l):44-45, 1974. Connellan, T.K. "Management Education 1970: The Current State of the Art" Mimeographed Paper for Seminar i n Adult Education, University of Michigan, 1970. Cote, D.P. "Measuring Results of Supervisory Training" Training and Development Journal 23(11):38-46, 1969. Curl, D.H. "Essentials of a Training System" Training  i n Business and Industry 4 (3):37-41, 1967. Darter, V.W. "County Extension Program Development" unpublished Doctor of Public Administration Thesis, Harvard University, 1955, c i t e d by E. deS. Brunner, et a l , An Overview of Adult Education Research Washington Adult Education Association of the U.S.A., 1959, p.135. Davis, J.N. "The Integration of Executive Development and Long-Range Planning" unpublished Ph.D. Thesis, Ohio State University, 1964. DePhillips, F.A., Berliner, W.M., and Cribbin, J.J. Management of Training Programs Homewood, I l l i n o i s : Richard D. Irwin, 1960. Duce, N.S. "On Setting Up Training Program" (Management Memo) The Province Newspaper Wednesday, May 12, 1971, Vancouver, B.C. 100 26. Edwards, A.L. Techniques of Attitude Scale Construction New York: Appleton-Century-Crofts, 1957. 27. Eitington, J.E. "The Training Function - Current and Future Directions" i n K.T. Byers, Employee Training and  Development i n the Public Service Chicago: Public Personnel Association, 1970. 28. Finn, M. "The Textbook Approach to Training" Journal of the American Society of Training Directors 13(5):17-20, 1959. 29. Hannon, W.J. "How Companies Look at Training" Training and Development Journal 22(l):32-34, 1968. 30. Houle, CO. The Design of Education San Francisco: Jossey-Bass, 1972. 31. House, R.J. Management Development: Design, Evaluation and Implementation Ann Arbour: Bureau of I n d u s t r i a l Relations, University of Michigan, 1967. 32. Keys, B.A., Thompson, F.G., and Heath, M. Meeting Managerial Manpower Needs Ottawa: Economic Council of Canada, 1971. 33. Kirkpatrick, D.L. "Techniques for Evaluating Training Programs" Journal of the American Society of Training  Directors Parts I to IV, Vol. 13, Nos. 11 and 12; Vol. 14, Nos. 1 and 2, 1959 and 1960. 34. Knowles, M.S. The Modern Practice of Adult Education New York: Association Press, 1970. 35. Kozoll, C.E. "The A i r Left the 'Bag' - A Training Program That Fai l e d " Training and Development Journal 25(7): 22-25, 1971. 36. LaForest, J.R. A Model for Program Planning i n Adult Education Atlanta, Georgia: Southern Regional Education Board, 1973. 37. Lawrie, J.W. and Boringer, C.W. "Training Needs Assessment and Training Program Evaluation" Training and  Development Journal 25(ll):6-9, 1971. 38. McGehee, W. and Thayer, P.W. Training i n Business and Industry New York: John Wiley & Sons, 1961. 101 39. McNary, J . "Special Training Situations" i n K.T. Byers, Employee Training and Development i n the Public Service Chicago: Public Personnel Association, 1970. 40. Markwell, D.S. and Roberts, T.J. Organisation of Management Development Programmes London: Gower Press, 1969. 41. M i l l e r , R.D. "A Systems Concept of Training" Training and Development Journal 23(4):4-14, 1969. 42. Nadler, L. "Support Systems for Training" Training and Development Journal 25(10):2-7, 1971. 43. Off, C.B. and Boutin, L.D. "Training Program Design". Training and Development Journal 21(8):20-33, 1967. 44. Powers, R.C. "Degress to Which an Iowa's County Extension Program Planning Process Met Selected C r i t e r i a " unpublished M.S. Thesis, Iowa State University, 1960. 45. Quinn, A.K. "In Training, the System's the Thing" Training and Development Journal 24(2):25-29, 1970. 46. Renton, M.B. "Developing In-Company Training Courses" Training and Development Journal 23(9):10-16, 1969. 47. Rose, H.C. The Development and Supervision of Training Programs New York: American Technical Society, 1964. 48. Shaffer, D.E. "Control Through Measurement: Meeting Objectives as a Means to J u s t i f y Training" Training  Director 1s Journal 18(9):39-50, 1964. 49. Singer, E.J. Training i n Industry and Commerce London: Institute of Personnel Management, 1969. 50. This, L. "Results-Oriented Training Designs" Training and Development Journal 25(4):8-14, 1971. 51. Tracey, W.R. Designing Training and Development Systems New York: American Management Association, 1971. 52. Evaluating Training and Development Systems New York: American Management Association, 1968. 53. Turner, B.T. The Organization and Management of Company Training London: In d u s t r i a l and Commercial Techniques, 1969. 102 54. Turpin, G.H. "Manager Training" Training and Development Journal 24(l):28-30, 1970. 55. Vetter, E.W. Manpower Planning for High Talent Personnel Ann Arbour: Bureau of I n d u s t r i a l Relations, University of Michigan, 1967. 56. Warren, M.W. Training for Results: A Systems Approach to the Development of Human Resources i n Industry Reading, Mass: Addison-Wesley, 1969. 57. Winer, B.J. S t a t i s t i c a l P r i n c i p l e s i n Experimental Design New York: McGraw-Hill, 1962. 103 APPENDIX A THREE EXERCISES USED WITH TRAINING JUDGES IN THE SELECTION OF STATEMENTS FOR THE INDEX OF PROGRAM PLANNING EXERCISE ONE JUDGEMEJrrS- OP ITEM VALIDITY 1 0 4 This exercise is part of a research project focusing on the procedures for planning and implementing training programs for managers and supervisors in business and industrye Seven major aspects of pilanning are being considered; 1. Corporate Analysis, 'and. Manpower Planning 2. Determining and Assessing Training Needs 3 . Designing the Training Program U* Legitimization 5. Instructional Support 6. Evaluation ?. Maintenance of Behavior. For each of the seven major planning aspects,, eight examples have been selected vhich are intended to illustrate behaviors! which might be performed by a company in planning and implementing training programs for i t s managers.' and supervisors. The purpose of the present exercise is to determine whether a group of training experts considers the chosen examples to be; valid illustrations! of the categories which the items are intended to represent. Approximately twenty training experts from across Western Canada have been requested to participate in the exercise. The exercise consists of seven sections, corresponding to the seven major aspects of program planning which are under consideration. Each section commences with a definition of the planning aspect being dealt with. The definition is followed by eight statements which ara intended to illustrate possible behaviors which might represent company action in performing the planning aspect. You are requested to consider each statement i n turn and to judge its. worth as an illustration of the planning aspect which i t has been intended to represent. In the blank space following each statemjnt you are requested to write an appropriate mmbsr from 1 to 5, indicating to which of the following opinions your judgement most closely corresponds: 1. This statement is NOT A VALID example of the planning aspect which i t is- intended to represent. 2. This statement i s a POOR example of the planning aspect which i t i s intended to represent, 3» This statement is an ADEQUATE example of the planning aspect which i t ie intended to represent. 4.. This statement is a GOOD example of the planning aspect which i t ic intended to represent. 5* This statement i s an EXCELLENT example of the planning aspect which i t i s intended to represent. CORPORATE ANALYSIS AND MANPQilER PLANNING 105 A study of company objectives, plans, and .structure and of the company'o re l a t i o n s h i p t o changes i n society f o r the purpose of estimating how the company might change and what kinds of personnel might be required f o r the company to succeed i n the future, -• ' 1. The o v e r a l l objectives of. -.the-.company are indicated i n a w r i t t e n document which i s kept up-to-date and which i s available to employees who are responsible' f o r planning t r a i n i n g programs© 2. The company i s engaged i n planning projects designed to estimate future ..-company manpower requirements. % The company attempts i n some formal manner to coordinate^ a l l h i r i n g , t r a i n ! ig, and promotions with studies of future manpower requirements. 4» The company i s engaged i n projects designed to study and speculate on fature r o l e s and r e s p o n s i b i l i t i e s of i t s employees. % A l i s t of retirement dates of managers and supervisors i s maintained up-to-date by the company as pari of i t s manpower planning operations. 6, The company establishes plans f o r 5 to 10 year's i n the future and those plan.3 are made known to employees who are responsible f o r planning t r a i n i n g programs. 7» The company employs at le a s t one person whose primary function i s to study future developments i n society and to report on t h e i r significance f o r the company. 8. The company i s engaged i n planning projects designed to speculate, study, or recoiunend p o t e n t i a l organisational changes or developments. JETERKENING AND ASSESSING TRAINING NEEDS A study of. company operations and company personnel f o r the purpose of determining where problems e x i s t at present or w i l l l i k e l y e x i s t i n the future, and an assessment of the p o t e n t i a l r o l e s of t r a i n i n g i n helping t o overcome these problems. 9. A study was performed of employee morale, effectiveness of communication, old irate f o r innovation, degre3 of i n t e r n a l c o n f l i c t , or degree of cominitnient to corporate objectives w i t h i n the company fo r the purpose of determining i f t r a i n i n g was necessary. 10* The work records of managers arid supervisor 3 c r t h e i r subordinates were analyzed for. such indicators as waste :-aaterials, r e j e c t s , re-rans, safety, sales, or other measures o.:* job peri"ormance to determine i f t r a i n i n g was necessary. N 11. The personnel records of. managers and supervisors or t h e i r subordinates were analyzed for.such i n d i c a t o r s as absenteeism, lateness, turnover, grievances, demotions,- Or requests f o r transfer f o r the purpo&a of determining i f training, ./as necessary. 12. Company managers and supervisors were asked f o r t h e i r opinions as t o the needs f o r t r a i n i n g i n the company., 13i> An up-to-date company inventory of the knowledge, s k i l l s , a t t i t u d e s , or interests of employees was used to determine i f t r a i n i n g wa3 necessary. 14* A job analysis and/or task analysis was performed to determine the necessary knowledge, attit u d e s , or s k i l l s required f o r managerial or supervisory jobs wi t h i n the company. 15» When problems ./ere located i n company operai-ions a study was performed to assess the l i k e l y e f f e c t s of t r a i n i n g as a po t e n t i a l a c t i o n to solve the problems. 16« The on-the-job performance of managers and supervisors was rated or judged to'ditermine i f t r a i n i n g was necessary. DESIGNING THE TRAINING PROGRAM Planning and designing a sequence of i n s t r u c t i o n a l a c t i v i t i e s to achieve the desired t r a i n i n g goals and then producing necessary program materials and making preparations for implements.ticn of the program w i t h i n the company. 17. The i n s t r u c t i o n a l s i t u a t i o n was designed t o simulate a c t u a l working conditions as much as possible so that behaviors learned i n the program could e a s i l y be transferred to the work s i t u a t i o n . IB* A l l i n s t r u c t i o n a l materials which were necessary f o r the program were designed before the program was formally began. 19. The'program i n s t r u c t o r wa3 active i n helping to design the sequence of i n s t r u c t i o n a l events. 20» The program watf designed s p e c i f i c a l l y f o r a group of employees whose working ;situation and t r a i n i n g needs v-^re known to the program designors,. 107 21. The objective:- of the training program were stated in'behavioral terms which indicated the appropriate behavior to be learned, the standard of performance to be "met, and the conditions under which the performance wt.s to take pla.ce. >• • ; ; • 22. Information ws s gathered to determine when participants were relatively free from heavy work pressures and the timetable for the train - 1 ng program was developed on t'-ie basis of that information. 23. Lesson plans cr meeting procedures were put through pilot tests and were revised wherever necessary before the program was formally begun. 24. The instruments and devices for the assessment of the program and it s participarts were designed before the program was formally began. LEGITIMIZATION Gaining the approval of company management for the training program, including management agreement that the program i s financially worthwhile, that the content of tie program is acceptable, and that the procedures for conducting the program are within company regulations. 25. A series of cost/benefit studies of alternative approaches to the training progrim was used to convince management of the efficient use of funds by the proposed training programs. 26. Approval for the program was facilitated by g i v i n g management an opportunity at different points in the planning process to suggest - changes which would bring the program more into line with company objectives and expectations. 27. A separate budget for the training program v/as exhibited for management approval. 28. Short preview programs were provided for the approval of top management before the f u l l scale program was begun. 29. Management demands for cost controls were partly met by calculating both a variable and a fixed cost component for the training program. 30. A formal presentation of the program proposal was offered to management in which the values of the training for the company * were clearly outlined. 31. A return on investment was calculated for t h 3 program in order to convince management that the program was of value to the company. 108 32. To f a c i l i t a t e program approval, management ^as asked to participate i n the development of the program. INSTRUCTIONAL SOrTOxT The provision of services ana personnel to Cacilitate the instructional process as the training program i s underway. 33* When specific resource people from within the company were requested for the training program by the instructor the company attempted to cbtain them and make arrangements for their release from other duties. 34» Adequate reference materials relating to tho program of instruction were available i n suff i c i e n t quantities for a l l participants. 35. The instructor was provided with sufficient secretarial assistance to meet his requirements before, during, and after the instructional phase of the training program. 36. A l l necessary audio-visual equipment needed for the program was provided by th'? company either through rental or ownership. 37. The training program was conducted i n an art a which was s p e c i f i c a l l y allocated to the program by the company, and i n which no other company a c t i v i ;ies were allowed to intrude curing the conduct of the program.. 38. Any special arrangements for food services £.t coffee breaks, luncheon meetings, or dinner meetings were handled by company employees other than the instructor. 39* Any necessary arrangements or bookkeeping details for f i e l d t r i p s , special meetings, conferences, etc. were handled by company employees other than the instructor, leaving the instructor free of such problems. 40. At least one member of the planning group for the program assisted the instructor throughout the instructional phase i n making any necessary adjustments i n the training program. . . 109 EVALUATION The-assessment of the training program informs of the effectiveness of the training system which produced i t , or in, te:nhs of the achievements of the participants, or in terms of resultant effects of the program on the company as a whole, 4-1* Participants were evaluated at several stages i n the program to determine thai r pi-ogress toward achieving enrrse objectives. 42. The instructional ability of the instructor or group leader was formally evaluated on the basis of his performance during the program. 43. The objectives- of the program were used as r. basis from which to evaluate the learning and achievements of the participants. 44. Before and after measures were made among the participants and among a similar group of employees not beinj'j trained with respect to objectives of the program for the purposu of determining the effects of the; program, 45. The on-the-job behavior of participants was observed or measured both before and after the program to evaluate the effects of the training program. 4.6. Participants were tested before the program was fore-ally begun to determine their starting levels of knowledge, skills, or attitudes which were relevant to the course objectives, so that the effects of the program could be evaluated more readily. 47. The planning procedures which were used to develop the program were evaluated to determine i f improvements could have been made in the planning processes. 48, A systematic schedule of evaluations was planned for use at intervals after the program was completed at? a means of checking on the retention levels of newly-learned behaviors over periods of time. MAINTENANCE OF BEHAVIOR Attempting to ensure that behaviors learned in the training program are carried over into the work situation. 49. Short refresher programs were provided for participants at intervals after the original program was completed. The superior c f each.participant was requested to attend a short informative program-which outlined the behaviors to be learned by program par t i c i p a n t s , so that;, the superior was aware of what to expect from h i s subordinate on-the-job. Some form of personal counseiiiig was provid *d f o r p a r t i c i p a n t s who experienced d i f f i c u l t y i n tr a n s f e r r i n g the t r a i n i n g behaviors to the job s e t t i n g a f t e r the program was coiapleted. The superior of each participant was requested to p o s i t i v e l y reinforce the newly-learned behavior on-the-job. Top management issued i n s t r u c t i o n s that behaviors learned i n the t r a i n i n g program were expected to be used on-the-job a f t e r the completion of the program. The environment w i t h i n which the participants work was analyzed to determine i f the newly-learned behaviors would be encouraged and supported so that those behaviors would continue to be used on-the-job. Participants who had problems i n implementing the newly-learned behaviors on-the-job were provided with s p e c i a l a d d i t i o n a l t r a i n i n g sessions. The superior cf each program participant war.- requested to submit a report to either the t r a i n i n g s p e c i a l i s t s or to top management o u t l i n i n g the extent t c which the newly-learned behaviors were used on-the-job by the part i c i p a n t a f t e r the completion of the program. EXERCISE TWO INSTRUCTIONS FOR THE CATEGORIZATION EXERCISE ' 111 This exercise i s part of a research project' focusing on the procedures f o r planning and imp]scenting t r a i n i n g programs•for managers and supervisors i n business and industry. Seven major .-aspects of program planning are being considered: 1. Corp:-rate Analysis and Manpower - Planning 2. Determining and Assessing Training Meads 3. Designing the Training Program 4.. l e g i t i m i z a t i o n 5. I n s t r u c t i o n a l Supjort 6. Evaluation 7. Maintenance of Behavior. I t i s possible f o r a company to perform any cf the seven major aspects o f planning i n a number of. d i f f e r e n t ways and for purposes of t h i s research project several examples of possible behaviors f o r each of the major planning aspects have been de-vised. The purpose of t h i s exercise i s to t e s t whether the examples of program planning behaviors which have been devised f o r the research project are c l e a r l y ' examples of the planning aspects which they are s p e c i f i c a l l y meant to i l l u s t r a t e . The exercise ±3 not meant t o be d i f f i c u l t and i t contains no t r i c k questions. There are no " r i g h t " or "wrong" answers on the part of any judge. The c r i t i c a l issue l i e s i a the agreement or lack of agreement among the group of judges f o r each statement. The exercise i s being completed by approximately twenty corporate t r a i l i n g s p e c i a l i s t s across Western Canada at the present t i : ; . 3 . The purpose i s to det jriaine whether a group of t r a i n i n g experts, a f t e r .reading a statement, can agre* on i t s l i k e l y function i n the program planning process f o r t r a i n i n g i n business and industry. Only those statements upon which there Is clear agreement from the group of judges w i l l be retained f o r further procedures i n the research projecto TASK It Please take the sheet l a b e l l e d "Definitions f o r Major Categories" and c a r e f u l l y read eajh of the seven d e f i n i t i o n s . TASK 2: Please take the f i v e pages of Statements to be Categorized. Each o f the statements i s neant to represent an a c t i v i t y or behavior which a company might perform i n c a r r / i n g out one of the seven planning aspects which have been defined f o r t h i s project. You are requested to write a number from 1 to 7 on the l i n e beside each statement, i n d i c a t i n g which of the planning aspects you f e e l the statement best represents. DEFINITIONS FOR MAJOR CATEGORIES 112 CORPORATE ANALYSIS AND-MANPOWER PLANNING . - * • A study of company, objectives,' plans, and structure and of the company's r e l a t i o n s h i p .to -jhanges i n society f o r the purpose of estimating how the /jcmpany might change and what kinds of personnel might be. required f o r the company to succeed i n t h e f u t u r e . DETERMINING AND ASSESSING TRAINING NEEDS A study of company operations und company personnel f o r ths-purpose of determining where problems ex±3t a t present or w i l l l i k e l y e x i s t i n the future, and an assessment of the p o t e n t i a l r o l e s of t r a i n i n g i n helping to overcome these proalerase EE SIGNING TILS TRAINING PROGRAM Planning and designing a sequence of i n s t r u c t i o n a l a c t i v i t i e s t o achieve the desired t r a i n i n g goals, and then producing njcessary program materials anc. making preparations f o r implementation of the program v i t h i n the company. LEGITIMIZATION Gilining the approval of company management f o r the t r a i n i n g program, including management agreement that the program i s f i n a n c i a l l y worthwhile, that the content of the program i s acceptable, end that the procedures f o r conducting the program are w i t h i n company regulations. INSTRUCTIONAL SU1F0RT The prov i s i o n of services and personnel t o f a c i l i t a t e the i n s t r u c t i o n a l process as the t r a i n i n g program i s underway. EVALUATION Tie assessment of the t r a i n i n g program i n terms of the effectiveness of the t r a i n i n g system which produced i t , or i n terms of the achievements of the pa r t i c i p a n t s , or i n terms of resultant e f f e c t s of the program on the company as a whole. MAINTENANCE OF BEHAVIOR Attempting t o ensure that behaviors learned i n the t r a i n i n g program are c a r r i e d over i n t o the work s i t u a t i o n . STATEMENTS TO HE GATE GO) LI ZED STATEMENT The instruments and devices f o r ; the assessment of the program and i t s participants were designed before v,ne' program was formally began. The on-the-job performance:-of..managers and supervisors, was rated or judged to determine i f t r a i n i n g wa-s necessary. A systematic schedule of evaluations was planned f o r use at i n t e r v a l s a f t e r the.program was completed an a means of checking on the retention l e v e l s of newly-learned behaviors, over periods of time. Lesson plans o.« meeting procedures were put through p i l o t t e s t s and were revised wherever necessary before the program was formally begun. When problems were located i n company operations a study was performed t o assess the l i k e l y e f f e c t s c i " t r a i n i n g as a po t e n t i a l a c t i o n to solve the problems. To f a c i l i t a t e program approval, management vas asked to p a r t i c i p a t e i n the developn-ent of the t r a i n i n g jrogram. Information was gathered to determine when panticipants were r e l a t i v e l y free from heavy work pressures and the timetable fo r the t r a i n i n g program was developed on the basis of that information. The superior o:* each program par t i c i p a n t was requested to submit a report to ei t h e r the t r a i n i n g spacialie bs or to top management t n i t l i n i n g the extent t c which the newly-learned behaviors were used on-the-job by the participant a f t e r the completion of the program. A return on investment was calculated for t h 3 program i n order to convince management that the program was of value to the company. The. planning procedures which were used to develop the program were evaluated t o determine i f improvements could have been made i n the planning processes. A formal presentation of the program proposal was offered t o management in while!-. the values of the t r a i n i n g f J T the company were c l e a r l y outlined., At least one member of the planning group for the program assisted the instructor throughout -the instinictidhf'.l phase i n making necessary adjustments in the', training progrjuai Participants who had'' problems -in implementing t i i a newly-learned behaviors on-the-job were provided with special additional training sessions. The company is engaged i n planning projects designed to speculate, study, or recommend potential organizational changes or developments. Any necessary arrangements or bookkeeping details for field trips, special meetings, conferences,, etc. were handled by company employees other than the instructor for the program, leaving the instructor free of such problems. The objectives of the training program were stated i n behavioral terms which indicated the appropriate behavior to be learned, the standard of performance tc be met, and the conditions under which the performance was to take place. The environment within which the participants work was analyzed to determine i f the newly-learned behaviors would be supported and encouraged so that those behaviors wcaild continue to be used on-the-job,. Any special arrangements for food services at coffee breaks, luncheon meetings, or dinner meetings were handled by company employees othf.jr than the instructor. The company employs at least one person whose primary function is to study future developments i n society and to report on their significance for the company* A job analysis and/or task analysis was performed to determine the necessary knowledge, attitudes, or sld.113 required for managerial or supervisory jobs within the company. Management demands for cost controls were partly met by calculating both a variable and a fixed cost component for the training program. Participants were tested before the program was formally begun to determine their starting levels of knowledge, skills, or attitudes which were relevant to the course objectives, so that the effects of the program could, be evaluated more readily. An up-to-date inventory of. the knowledge, skills, attitudes, or interests of employees was used to determine i f training was necessary. Top management issued instructions that behaviors learned i n the training program were expected to be used on-the-job after the completion of the program. • • ' '>. The company establishes plans for 5 to 10 years in the future and those plans are made known to employees who are responsible for planning training programs. The program wa3 designed specifically for a group of employees whose working situation and training neec s were known to the program planners. The training program was conducted i n a location which' was specifically allocated to the program by the company, and no other company activities were allowed to intrude i n the location during the conduct of the program. A l i s t of retirement dates of managers and supervisors i s maintained up-to-dite by the company as part of its. manpower planning operations. The on-the-job behavior of participants was observed or measured both before and after the program to evaluate the effects of the training program. The company is engaged in projects designed to study and speculate on future roles and responsibilities of i t s employees. A l l necessary audio-visual equipment needed for the program was. provided by the company either through rental or ownership. Short preview programs were provided for the approval of top management before the f u l l scale program was began. The company attempts in some formal manner to coordinate a l l hiring, training, and promotions with studios cf future manpower requirements* A separate budget for the training program was exhibited for management approval. The company i s engaged i n planning projects designed to estimate future company manpower requirements. Before and a f t e r measures were made among the participants and among a s i m i l a r group of employees not being t r a i n e d with respect to objectives of the program f o r the purpose of determining the e f f e c t s of the program^ - The superior of each participant was requested to p o s i t i v e l y reinforce the newly-learned behavior on-the-job. The o v e r a l l objectives of the company are indicated i n a w r i t t e n document which i s kept up-to-date and which i s available t o employees who are responsible f o r planning t r a i n i n g programs* Approval f o r the program was f a c i l i t a t e d by giving management an opportunity at d i f f e r e n t points i n the planning process to suggest changes which would bring the program more i n t o l i n e with company objectives and expectations* Some form of personal counseling was provided f o r participants who experienced d i f f i c u l t y i n t r a n s f e r r i n g the t r a i n i n g behaviors t o the job s e t t i n g a f t e r the program was completed. The objectives of the program were used as a basis from which to evaluate the learning and achievements of the p a r t i c i p a n t s . Company managers and supervisors were asked for t h e i r opinions as to the needs f o r t r a i n i n g i n the company. The i n s t r u c t o r was provided with s u f f i c i e n t s e c r e t a r i a l assistance to meet h i s requirements before, during, and a f t e r the i n s t r u c t i o n a l phase of the t r a i n i n g program. Short refresher programs were provided for participants at i n t e r v a l s a f t e r the o r i g i n a l program was completed. The program i n s t r u c t o r was active i n helping to design the sequence of i n s t r u c t i o n a l events. The superior of each par t i c i p a n t was requested to attend a short informative program which outlined the behaviors t o be learned by program par t i c i p a n t s , so that the superior was aware of what to expect from h i s subordinate on-the-job. A se r i e s of cost/benefit studies of a l t e r n a t i v e approaches t o the t r a i n i n g program was used to convince management of the e f f i c i e n t use of funds by the proposed t r a i n i n g program. The. i n s t r u c t i o n a l a b i l i t y of the i n s t r u c t o r or group leader was formally evaluated oh the basis of h i s performance-during the-program. The personnel records of..,sahagers and supervisors .or t h e i r subordinates were analysed f o r such in d i c a t o r s as Absenteeism, lateness, turnover, r grievances, demotions, or requests f o r transfer f o r the purpose of determining i f t r a i n i n g was necessary. A l l i n s t r u c t i o r i a l . materials which were necessary f o r the program were designed before the program was formally begun. Adequate reference materials r e l a t i n g to the program of i n s t r u c t i o n were a v a i l a b l e i n s u f f i c i e n t quantities, f o r the part i c i p a n t s . The work records of managers and supervisors or t h e i r subordinates were analysed f o r such i n d i c a t o r s as vaste materials, r e j e c t s , re-runs, safety, sales, or other measures of job performance to determine i f t r a i n i n g was necessary. When s p e c i f i c resource people from w i t h i n t i e company were requested f o r tho t r a i n i n g program by the in s t r u c t o r the company attempted to obtain them and make arrangements f o r t h e i r release from other duties. A study was performed of employee morale, effectiveness of communication, c l i : a t e f o r innovation, degree of i n t e r n a l c o n f l i c t , or degree of commitment to corporate objectives w i t h i n the company f o r the p.irpose of determining i f t r a i n i n g was necessary. The ins t r u c t i o ; ^ a l s i t u a t i o n was designed to simulate a c t u a l working conditions as much as possible so that the behaviors learned i n the program could e a s i l y be transferred to the work se t t i n g . P a r t icipants were evaluated at several stages i n the program to determine t h e i r progress toward achieving program objectives. EXERCISE THREE 118 QUESTIONNAIRE: THE IMPORTANCE OF INDIVIDUAL PLANNING ACTIVITIES FOR MANAGERIAL. TRAINING IN BUSEKS3 AND IMDUSTRI The major task in this. .questionnaire 1,3 to judge the relative importance of er ch activity in the following- list of statements as i t would contribute to the planning and implementation of a successful training program for managers of supervisors In business and industry. Naturally, the importance of activities varies i n different situations and types of programs. You are asked to rate the overall AVERAGE importance of the activity, considering the total range of possible programs which could be planned for managers und supei'Visors from within a company. Please read each statement in turn and judge i t s importance i n contributing to the planning and implementation of training programs as noted above. Indicate your opinion of importance by writing a number from 1 to 9 i n the blank space following the .'statement. The number should correspond to the following scale in wiiich greater importance i s indicated by higher numbers: OF NO EXTREMELY IMPORTANCE IMPORTANT 1 2 3 4 5 6 7 8 9 STATEMENTS OF PLANNING ACTIVITIES The overall objectives of the company are indicated in a written document which is kept up-to-date and which is available to employees who are responsible for planning training programs. 2.. The company is engaged in planning projects designed to estimate future company manpower requirements. 3. The company attempts in some formal manner to coordinate a l l hiring, training, and promotions with studies of future manpower requirements. U» The company is engaged in projects designed to study and speculate on future roles and responsibilities of i t s employees. 119 A l i s t of retirement'dates, of managers arid supervisors i s ^maintained up-to-date by the company a3 part of i t s manpower planning operations* The company establishes plans for 5 to 10 years i n the future and those* plans are made known to employees who are responsible for planning training programs. The compeny is engaged in planning projects designed to speculate, study, or recommend potential organisational changos or development. The work records of managers and supervisors or their subordinttas were analyzed for such iniicators as waste materials, rejects, re-runs, safety, sales, or other measures of job performance to determine i f training w.is necessary. The personnel records of managers and supervisors or their subordinates were analyzed for such iniicators as absenteeism, lateness, turnover, grievances, demotions, or requests for transfer for the purpose of determining i f trailing was necessary. Company managers and supervisors were asked for their opinions as to the needs for training in the conpany. An up-to-date company inventory of the knowledge, skills, attitudes, or interests of employees wt.s used to determine i f training was necessary. A job analysis and/or task analysis war. performed to determine the necessary knowledge, attitudes, or skills required for managerial or supervisory jobs within the company. • When problems were located i n company operations a study was performed to assess the likely effects of training as a potential action to solve the problems. The instructional situation was designed to simulate actual working conditions a3 much as possible so that behaviors learned i n the program could easily be transferred to the work situation. A l l instructional materials which were necessary for the program were designed before the program was formally bagun. The program instructor was active in helping to design tb.3 sequence of instructional events. 12 0 The program was designed specifically for a group of employees whose working situation and training needs were known to the program designers. Information was gathered to determine when participants were relatively free from heavy work pressures and the timetable f o r the training program was developed on the basis of that information. Lesson pilaris or meeting procedures were put through pilot te3ts and were revised wherever necessary before the program was formally begun. A series of cost/benefit studies of alternative approaches to the training program was used to convince 'Jianagement of the efficient use of funds by the proposed training program. Approval for the program was facilitated by giving management an opportunity at different points in the planning process to suggest changes vhich would bring the program more into line with company objective s and expectations. A separate budget for the training pro;*ram was exhibited for management approval. Short preview programs were provided for the approval of top management before the f u l l scale program was begun. Management demands for cost controls wore partly met by calculating both a variable and a fixed cost component for the training program, A formal presentation of the program proposal was offered to managemert in which the values of the tx-aining for the company were clearly outlined. A return on investment was calculated for the program i n order to convince management that the program was of value to the company. To facilitate program approval, management was asked to participate in the development of the program. When specific resource people from within the company were requested for the training program by the instructor the company attempted to obtain them and make arrangements for their release from other duties. 121 29 • The instinctor was pr'6yi:Asd with sufficient secretarial assistance to meet his requirements, before j during}, and after the instructional phase of the training' program. _ 30. A l l necessary audio-visual equipment heeded for the program was provided by the company either through rental or ownership, to 31* The trailing program was conducted in m area which \<ras specifically . allocated to the program by the company, and in which no other company cctivities were allowed to intrude during the conduct of the program. _ 32. Any necessary arrangements or bookkeeping details for field trips, special meetings, conferences, etc. we:?e handled by company employees other than the instructor, leaving the instructor free of such problems. 33. At least one member of the planning group for the program assisted the instructor throughout the instructional phase in making any necessary adjustments in the training program. (_ 3A» Participants were evaluated at several stages in the program to determine their progress toward achieving course objectives, ^ 35* The instructional ability of the instructor or group leader was formally evaluated on the basis of his performance during the program, 36. The objectives of the program veve used as a basis from which to evaluate the learning and achievements of the participants. 37. Before an! after measures were made among the participants and among a similar group of employees not being trained with respect to objectives of the program for the purpose of determining the effects 'of the program. 33. The on-the-job behavior of participants was observed or measured both before and after the program to evaluate the effects of the training program. 39» Participants were tested before the program was formally begun to determine their starting levels of knowledge, skills, or attitudes which were relevant to the course objectives, so that the effects of the program could be evaluated more readily. 4.0. The planning procedures which ware usee' to develop the program were evaluated to determine i f improvements could have been made in the pit 'lining processes. 122 Short refresher programs were provided for pai*ticipants at intervals after the original program was completfid'i The superior of each participant was. requested to attend a short informative program ^rnich outlined.the behaviors to be learned by program participants, so,that the superior was aware of what to aspect from his subordinate on-the-job,. The superior of each participant was requested to positively reinforce the newly-learned behavior o^i-the-job. Participants who had problems in implementing the newly-learned behaviors on-the-job were provided with special additional training sessions. Some form of personal counseling was provided for 1 participants who experienced difficulty in transferring the training behaviors to the job setting after the program was completed. APPENDIX B ORIGINAL DATA FROM THE FIRST AND SECOND EXERCISES 1 2 4 F R E Q U E N C Y D I S T R I B U T I O N S A N D M E D I A N V A L U E S F O R S T A T E M E N T S I N T H E V A L I D I T Y J U D G E M E N T E X E R C I S E S T A T E M E N T V A L U E S T O T A L M E D I A N 1 2 3 4 5 V A L U E . I N V A L I D A D E Q U A T E E X C E L L E N T 1 . 1 1 4 3 9 1 8 4 . 5 0 2 . 0 0 8 6 4 1 8 3 . 6 7 3 . 0 0 0 6 4 1 8 3 . 6 7 4 . 3 2 5 4 4 1 8 3 . 3 0 5 . 0 3 8 2 5 1 8 3 . 2 5 6 . 1 1 4 6 6 1 8 4 . 0 0 7 . 5 5 4 1 3 1 8 2 . 3 0 8 . 1 2 5 6 4 1 8 3 . 6 7 9 . 2 7 5 2 2 1 8 2 . 5 0 1 0 . 2 2 6 5 3 1 8 3 . 2 5 1 1 . 2 4 4 5 3 1 8 3 . 3 8 1 2 . 0 1 9 3 5 1 8 3 . 3 9 1 3 . 1 0 6 3 8 1 8 4 . 1 7 1 4 . 0 0 4 8 6 1 8 4 . 1 3 1 5 . 1 3 4 3 7 1 8 3 . 8 3 1 6 . 2 5 3 6 2 1 8 3 . 1 7 1 7 . 1 2 2 4 9 1 8 4 . 5 0 1 8 . 1 1 7 5 4 1 8 3 . 5 0 1 9 . 1 0 4 7 6 1 8 4 . 0 7 2 0 . 1 0 3 5 9 1 8 4 . 5 0 2 1 . - 0 0 3 3 12 1 8 5 . 0 0 2 2 . 1 2 6 4 5 1 8 3 . 5 0 2 3 . 0 1 8 4 5 1 8 3 . 5 0 2 4 . 0 4 4 5 5 1 8 3 . 7 0 2 5 . 2 1 3 4 8 1 8 4 . 2 5 2 6 . 0 2 6 3 7 1 8 3 . 8 3 2 7 . 0 2 5 7 4 1 8 3 . 7 9 2 8 . 1 4 5 4 4 1 8 3 . 3 0 2 9 . 2 3 4 7 2 1 8 3 . 5 0 3 0 . 0 1 2 6 9 1 8 4 . 5 0 3 1 . 3 1 4 5 5 1 8 3 . 7 0 3 2 . 1 1 5 3 8 1 8 4 . 1 7 3 3 . 0 0 3 6 9 1 8 4 . 5 0 3 4 . 0 0 3 2 1 3 1 8 5 . 0 0 12 5 S T A T E M E N T V A L U E S T O T A L M E D I A 1 1 2 3 4 5 V A L U E I N V A L I D A D E Q U A T E E X C E L L E N T 3 5 . 0 1 4 4 9 1 8 4 . 5 0 3 6 . 0 0 1 7 1 0 1 8 5 . 0 0 3 7 . 0 1 3 3 1 1 1 8 5 . 0 0 3 8 . 1 5 5 2 5 1 8 3 . 1 0 3 9 . 0 3 8 4 3 1 8 3 . 2 5 4 0 . 0 1 4 9 4 1 8 3 . 9 4 4 1 . 1 1 4 7 5 1 8 3 . 9 3 4 2 . 3 3 3 4 5 1 8 3 . 5 0 4 3 . 2 0 3 4 9 1 8 4 . 5 0 4 4 . 1 1 3 3 1 0 1 8 5 . 0 0 4 5 . 0 1 4 3 1 0 1 8 5 . 0 0 4 6 . 1 1 5 6 5 1 8 3 . 8 3 4 7 . 1 4 3 3 7 1 8 3 . 8 3 4 8 . 2 0 4 7 5 1 8 3 . 9 3 4 9 . 0 2 7 6 3 1 8 3 . 5 0 5 0 . 2 0 3 6 7 1 8 4 . 1 7 5 1 . 0 2 5 4 7 1 8 4 . 0 0 52 . 1 2 4 7 4 1 8 3 . 7 9 5 3 . 6 5 3 1 3 1 8 2 . 1 0 5 4 . 1 4 4 4 5 1 8 3 . 5 0 5 5 . 0 1 6 5 6 1 8 3 . 9 0 5 6 . 1 4 3 5 5 1 8 3 . 7 0 126 FREQUENCY DISTRIBUTIONS FOR STATEMENTS IN THE CATEGORIZATION JUDGEMENTS EXERCISE STATEMENT CATEGORIES TOTAL PER CENT IN 1 2 3 4 5 6 7 AGREEMENT WITH DESIGN CATEGORY 1. 13 2 0 1 0 0 0 16 81. 3 2 . 15 1 0 0 0 0 0 16 93.8 3. 16 0 0 0 0 0 0 16 100.0 4. 14 2 0 0 0 0 0 16.. 87.5 5. 15 1 0 0 0 0 0 16 93.8 6. 14 2 0 0 0 0 0 16 87.5 7. 16 0 0 0 0 0 0 16 100.0 8. 16 0 0 0 0 0 0 16 100. 0 9. 4 12 0 0 0 0 0 16 75.0 10. 0 15 0 0 0 1 0 16 93.8 11. 1 15 0 0 0 0 0 16 93.8 12. 0 15 0 1 0 0 0 16 93.8 13. 0 16 0 0 0 0 0 16 100.0 14. 6 10 0 0 0 0 0 16 62.5 15. 1 15 0 0 0 0 0 16 93.8 16. 1 15 0 0 0 0 0 16 93.8 17. 0 0 14 0 0 0 2 16 87.5 18. 0 0 14 0 2 0 0 16 87.5 19. 0 0 15 0 1 0 0 16 93.8 20. 0 2 14 0 0 0 0 16 87.5 21. 1 1 9 2 0 2 0 16 56.3 22. 1 1 13 0 1 0 0 16 81.3 23. 0 0 12 2 1 1 0 16 75.0 24. 2 1 3 0 0 10 0 16 18.8 25. 0 0 1 14 0 1 0 16 87.5 26. 0 0 6 10 0 0 0 16 62.5 27. 2 0 0 14 0 0 0 16 87.5 28. 1 0 2 13 0 0 0 16 81.3 29. 1 0 1 14 0 0 0 16 87.5 30. 1 2 0 13 0 0 0 16 81.3 31. 1 0 0 12 0 3 0 16 75.0 32. 2 1 1 12 0 0 0 16 75.0 127 STATEMENT CATEGORIES TOTAL PER CENT IN I 2 3 4 5 6 7 AGREEMENT WITH DESIGN CATEGORY 33. 0 0 2 1 13 0 0 16 81.3 34. 0 0 7 0 9 0 0 16 56.3 35. 0 1 1 0 14 . 0 0 16 87.5 36. 0 0 4 0 12 0 0 16 75.0 37. 0 0 2 0 14 0 0 16 87.5 38. 0 0 1 0 15 0 0 16 93.8 39. 0 0 1 0 15 0 0 16 93.8 40. 0 1 1 0 13 1 0 16 81.3 41. 0 0 2 0 0 13 1 16 81.3 42. 0 1 0 0 1 14 0 16 87.5 43. 0 0 0 0 0 16 0 16 100.0 44. 0 0 0 0 0 16 0 16 100.0 45. 0 0 0 0 0 15 1 16 93.8 46. 0 3 2 0 0 11 0 16 68.8 47. 1 0 4 0 0 11 0 16 68.8 48. 0 0 2 0 0 5 9 16 31.3 49. 0 0 1 0 0 0 15 16 93.8 50. 0 0 2 3 0 0 11 16 68.8 51. 0 0 0 0 0 1 15 16 93.8 52. 0 0 0 0 0 1 15 16 93.8 53. 2 0 0 .4 0 0 10 16 62.5 54. 1 3 2 1 0 1 8 16 50.0 55. 0 0 1 0 1 0 14 16 87.5 56. 0 0 0 0 0 7 9 16 56.3 12 8 APPENDIX C DATA ANALYSES RELATED TO EXERCISE THREE AND TO THE INDEX OF PROGRAM PLANNING 12 9 The purpose of t h i s appendix i s to outline the procedures of data analysis which were conducted at the conclusion of the t h i r d exercise. The exercise was designed to test the degree of agreement among a group of tra i n i n g judges with respect to the importance of f o r t y - f i v e selected program planning procedures. F i f t y - e i g h t judges used a nine-point scale i n ra t i n g the importance of each stated behaviour i n the planning process, with 1 representing an opinion of "no importance" and 9 representing an opinion of "extremely important". The frequency d i s t r i b u t i o n s which were derived from the opinions provided by the f i f t y - e i g h t judges are provided i n Table A. As may be seen, the majority of opinions for almost a l l of the f o r t y - f i v e statements were at a l e v e l of 5 or more on the nine-point scale. However, the lack of agreement may also be seen, as some judges rated many of the statements at the low end of the scale, so that opinions for any one statement were spread over seven, eight, or nine points on the scale, as a general r u l e . The o r i g i n a l frequency d i s t r i b u t i o n was not used for purposes of further data analysis because of possible i n e q u a l i t i e s of unknown size i n the widths of the nine categories. The judges were not instructed to consider the categories of equal size, and the variety of interpretations with respect to r e l a t i v e category size made i t necessary to transform the f o r t y - f i v e d i s t r i b u t i o n s onto an underlying continuum which was common to a l l statements and which consisted of i n t e r v a l widths which were known, even though they may not have been of equal si z e . The construction of an underlying continuum and the transformation of the f o r t y - f i v e separate d i s t r i b u t i o n s onto i t was completed using the method of successive i n t e r v a l s , as 130 TABLE A FREQUENCY DISTRIBUTIONS FOR JUDGEMENTS OF STATEMENT IMPORTANCE STATEMENT CATEGORIES OF IMPORTANCE TOTAL 1 2 3 4 5 6 7 8 9 1. 0 1 2 1 4 7 8 11 24 58 2. 0 1 2 7 1 7 18 9 13 58 3. 1 1 0 1 9 10 19 7 10 58 4. 0 1 5 2 10 12 17 6 5 58 5. 3 3 3 7 12 7 10 5 8 58 6. 0 2 3 3 5 11 11 16 7 58 7. 0 2 2 9 9 13 11 8 4 58 8. 2 2 0 5 6 10 18 8 7 58 9. 4 2 3 5 14 8 10 7 5 58 10. 0 2 0 0 4 9 13 14 16 58 11. 2 1 3 0 11 8 12 14 7 58 12. 2 0 2 2 4 7 12 15 14 58 13. 1 1 2 4 9 8 13 15 5 58 14. 1 0 2 0 4 7 12 21 11 58 15. 3 5 2 4 5 10 10 14 5 58 16. 1 1 2 1 2 6 14 17 14 58 17. 0 1 1 2 4 3 10 24 13 58 18. 1 4 3 3 4 8 15 13 7 58 19. 0 2 1 6 11 13 11 10 4 58 20. 1 0 3 6 16 12 6 7 7 58 21. 0 0 2 3 6 9 14 12 12 58 22 . 1 0 3 5 6 8 13 12 10 58 23. 1 9 3 5 7 15 10 6 2 58 24. 2 3 7 6 9 14 10 4 3 58 25. 0 0 4 2 7 5 13 18 9 58 26. 3 4 2 11 11 6 13 5 3 58 27. 1 3 4 3 3 8 17 12 7 58 28. 0 1 0 0 6 8 17 13 13 58 29. 0 3 2 3 7 11 16 9 7 58 30. 0 2 2 3 5 3 18 15 10 58 31. 0 1 4 3 1 4 5 21 19 58 32. 2 5 3 6 11 5 13 9 4 58 33. 2 1 3 6 6 13 15 11 1 58 34. 1 1 0 1 6 9 15 17 8 58 35. 2 2 2 3 6 5 12 16 10 58 131 STATEMENT CATEGORIES OF IMPORTANCE TOTAL 1 2 3 4 5 6 7 8 9 36. 1 0 0 0 5 2 11 20 19 58 37. 3 1 4 2 3 9 19 12 5 58 38. 2 1 3 1 5 2 12 21 11 58 39. 3 2 3 2 8 5 17 12 6 58 40.. 2 0 5 3 5 9 20 7 7 58 41. 0 0 0 3 9 10 15 14 7 58 42. 0 3 0 1 5 7 10 14 18 58 43. 1 0 1 2 4 3 14 14 19 58 44. 1 2 2 4 6 9 13 12 9 58 45. 1 0 0 5 7 4 12 18 11 58 outlined by Edwards i n Techniques of Attitude Scale Construction. The method required that the o r i g i n a l frequency d i s t r i b u t i o n for each statement be converted to a proportional d i s t r i b u t i o n , as may be seen i n Table B. That proportional d i s t r i b u t i o n was then successively summed across the categories from l e f t to r i g h t to produce a cumulative proportion d i s t r i b u t i o n , as i s shown i n Table C. At that point, an assumption was made that the cumulative proportion d i s t r i b u t i o n s were normal for each statement when they were projected onto the common continuum, which allowed the unit-normal curve and i t s well-known s t a t i s t i c a l properties to be used i n the development of the continuum. The z scores were obtained for each of the cumulative proportions exhibited across the categories for each statement. Those z scores were derived from the table of the unit-normal curve, and the re s u l t s are shown i n Table D. The differences i n terms of z scores were then calculated between each successive i n t e r v a l across each of the f o r t y - f i v e 132 TABLE B PROPORTIONAL DISTRIBUTIONS FOR JUDGEMENTS OF STATEMENT IMPORTANCE STATEMENT CATEGORIES OF IMPORTANCE TOTAL . 1 2 3 4 5 6 7 8 9 1. .00 .017 .034 .017 .069 .121 .138 .190 .414 1.00 2. .00 .017 .034 .121 • .017 .121 . . 310 . 155 .224 1.00 3. .017 .017 .00 . 017 . 155 .172 .328 .121 .172 1.00 4. . 00 .017 . 086 .034 .172 .207 .293 .103 .086 1.00 5. .052 .052 .052 .121 .2 07 .121 . .172 .086 .138 1.00 6. .00 .034 .052 .052 . 086 .190 .190 .276 .121 1.00 7. .00 .034 .034 .155 .155 .224 .190 .138 .069 1.00 8. .034 .034 .00 .086 .103 . 172 .310 .138 .121 1.00 9. . 069 .034 .052 .086 .241 .138 .172 .121 . .086 1.00 10. .00 .034 .00 .00 .069 .155 .224 .241 . .276 1.00 11. .034 .017 .052 .00 .190 .138 .207 .241 .121 1.00 12. .034 .00 . 034 .034 .069 .121 .207 .259 .241 1.00 13. .017 .017 .034 .069 .155 .138 .224 .259 .086 1.00 14. . 017 .00 .034 .00 . 069 .121 .207 .362 .190 1.00 15. .052 .086 .034 .069 .086 .172 .172 .241 .086 1.00 16. .017 .017 .034 .017 .034 .103 .241 .293 .241 1.00 17. .00 .017 .017 .034 .069 .052 .172 . .414 .224 1.00 18. .017 .069 .052 .052 .069 .138 .259 .224 .121 1.00 19. .00 .034 .017 .103 .190 .224 .190 .172 .069 1.00 20. .017 .00 .052 .103 .276 .207 .103 .121 .121 1.00 21. .00 .00 .034 .052 .103 .155 .241 . .207 .207 1.00 22 . .017 .00 .052 .086 .103 .138 .224 .207 .172 1.00 23. .017 .155 .052 .086 .121 .259 .172 .103 .034 1.00 24. .034 .052 .121 .103 .155 .241 . .172 .069 .052 1.00 25. .00 .00 .069 - .034 .121 .086 .224 . 310 .155 1.00 26. .052 .069 .034 .190 .190 .103 .224 .086 .052 1.00 27. .017 .052 . 069 .052 .052 .138 .293 .207 .121 1.00 28. .00 .017 .00 .00 .103 .138 .293 .224 .224 1.00 29. .00 .052 .034 .052 .121 .190 .276 .155 .121 1.00 30. .00 .034 .034 .052 .086 .052 .310 .259 .172 1.00 31. .00 .017 .069 .052 .017 .069 .086 .362 .32 8 1.00 32. .034 .086 .052 .103 .190 .086 .224 .155 .069 1.00 33. .034 .017 .052 .103 .103 .224 .2 59 .190 .017 1.00 34. .017 .017 .00 .017 .103 .155 .259 .293 .138 1.00 35. .034 .034 .034 .052 .103 .086 .207 .276 .172 1.00 133 STATEMENT CATEGORIES OF IMPORTANCE TOTAL 1 2 3 4 5 6 7 8 9 36. .017 .00 .00 .00 . 086 .034 .190 . 345 . 32 8 1.00 37. . 052 .017 .069 .034 .052 .155 .32 8 .207 .086 1.00 38. .034 .017 .052 .017 .086 .034 .207 . 362 . 190 1.00 39. .052 .034 .052 .034 .138 .086 .293 .207 .103 1. 00 40. .034 .00 .086 .052 .086 .155 .345 . 121 .121 1.00 41. . 00 .00 .00 .052 .155 .172 .259 .241 .121 1.00 42 . .00 .052 .00 .017 .086 .121 .172 .241 . 310 1.00 43. .017 .00 .017 .034 .069 .052 .241 .241 .32 8 1.00 44. ,017 .034 .034 .069 .103 .155 .224 .207 .155 1.00 45. .017 .00 .00 .086 .121 . 069 .207 . 310 .190 1.00 di s t r i b u t i o n s , and those r e s u l t s are indicated i n Table E. In some instances the lower category i n t e r v a l s did not include z scores since the d i s t r i b u t i o n commenced at a higher l e v e l on the scale, and therefore, no difference i n z scores was calculated. For each column of category interv a l s , an average difference with respect to z scores was calculated, considering the t o t a l number of z score differences which were included i n the column. The c a l c u l a t i o n may be seen at the bottom of Table E, where the sum of i n t e r v a l s and number of interv a l s were used to calculate the mean i n t e r v a l width, i n terms of z scores. Those average widths were then summed successively from l e f t to r i g h t to produce the common underlying continuum which was the objective of the procedure. The common underlying continuum for the project may be seen at the bottom of Table. E, and i t i s la b e l l e d "Cumulative Width". The continuum commences at Interval One, with a value of 0.0 at the top of the f i r s t i n t e r v a l . The value at the top of 134 TABLE C CUMULATIVE PROPORTIONS FOR JUDGEMENTS OF STATEMENT IMPORTANCE STATEMENT CATEGORIES OF IMPORTANCE 1 2 3 4 5 6 7 8 9 1. .00 .017 .052 .069 .138 .259 : .397 .586 1.00 2. .00 .017 .052 .172 .190 .310 .621 .776 1.00 3. .017 .034 .034 .052 .207 .379 .707 .82 8 1.00 4. . 00 .017 .103 .138 .310 .517 .810 .914 1.00 5. .052 .103 .155 .276 .483 .603 .776 .862 1.00 6. .00 .034 .086 .138 .224 .414 .603 .879 1.00 7. . 00 .034 .069 .224 .379 .603 .793 .931 1.00 8. .034 .069 .069 .155 .259 . .431 . .741 . . 879 1.00 9. .069 .103 .155 .241 . .483 .621 .793 .914 1. 00 10. .00 .034 .034 .034 .103 .259 .483 .724 1. 00 11. .034 .052 .103 .103 .293 .431 .638 .879 1.00 12. .034 .034 .069 .103 .172 .293 .500 . 759 1.00 13. .017 .034 .069 .138 .293 .431 .655 .914 1.00 14. .017 .017 .052 .052 .121 .241 . .448 .810 1.00 15. .052 .138 .172 .241 .328 .500 .672 .914 1. 00 16. .017 . .034 .069 .086 .121 . .224 .466 .759 1.00 17. .00 .017 .034 .069 .138 .190 .362 . 776 1.00 18. .017 .086 .138 .190 .259 .397 .655 . 879 1.00 19. .00 .034 .052 .155 .345 .569 .759 .931 1.00 20. .017 .017 .069 .172 .448 .655 .759 . .879 1.00 21. .00 .00 , .034 .086 .190 .345 .586 .793 1.00 22. .017 .017 .069 .155 .259 .397 .621 .828 1.00 23. .017 .172 .224 .310 .431 . .690 .862 .966 1.00 24. .034 .086 .207 .310 .466 .707 .879 . .948 1.00 25. .00 .00 .069 .103 .224 .310 .534 .845 1.00 26. .052 .121 . .155 .345 .534 .638 .862 .948 1.00 27. .017 .069 .138 .190 .241 .379 .672 .879 1.00 28. .00 .017 .017 .017 .121 .259 . .552 .776 1.00 29. .00 .052 .086 .138 .259 .448 .724 .879 1.00 30. .00 .034 .069 .121 .207 .259 .569 .828 1.00 31. .00 .017 .086 .138 .155 .224 .310 .672 1.00 32. .034 .121 .172 .276 .466 .552 .776 .931 1.00 33. .034 .052 .103 .207 .310 .534 .793 .983 1.00 34. .017 .034 .034 .052 .155 .310 .569 .862 1.00 35. .034 .069 .103 .155 .259 .345 .552 .82 8 1.00 135 STATEMENT CATEGORIES OF IMPORTANCE 1 2 3 4 5 6 7 8 9 36. .017 .017 . 017 .017 .103 .138 .32 8 .672 1.00 37. .052 .069 .138 .172 .224 .379 .707 . .914 1.00 38. .034 .052 .103 .121 .207 .241 . .448 .810 1.00 39. .052 .086 .138 .172 .310 .397 .690 .897 1.00 40. .034 .034 .121 .172 .259 .414 .759 .879 1.00 41. .00 .00 . .00 .052 .207 .379 .638 .879 1.00 42. .00 .052 .052 .069 .155 .276 .448 .690 1.00 43. .017 .017 .034 .069 .138 .190 .431 . .672 1.00 44. .017 .052 .086 - .155 .259 .414 .638 .845 1.00 45. .017 .017 .017 .103 .224 .293 .500 .810 1.00 the second i n t e r v a l i s 0.32 7, at the top.of the t h i r d i n t e r v a l i s 0.583, and so forth, up to the top of the eighth i n t e r v a l , where the value i s 3.052. The ninth i n t e r v a l was eliminated from the calculations, because the cumulative proportion to that i n t e r v a l was constantly 1.00, and the z score for that value i s i n f i n i t y . For purposes of the study, not a l l points on the o r i g i n a l d i s t r i b u t i o n were transformed onto the underlying continuum which was constructed. Only two pieces of information were required from each of the f o r t y - f i v e d i s t r i b u t i o n s , and they were the median value of importance and the semi-interquartile range of the d i s t r i b u t i o n . Because the median value was defined as the location of the f i f t i e t h percentile i n the d i s t r i b u t i o n , and the semi-interquartile range was defined as the difference i n scale values between the seventy-fifth percentile and the twenty-fifth percentile, only three values were required for each d i s t r i b u t i o n on the common continuum: the three percentile locations, l a b e l l e d P„._, Pc., P-,,--136 TABLE D UNIT NORMAL DEVIATES CORRESPONDING TO CUMULATIVE PROPORTIONS FOR JUDGEMENTS OF STATEMENT IMPORTANCE STATEMENT CATEGORIES OF IMPORTANCE 1 2 3 4 5 6 7 8 1. -1.63 -1.49 -1.09 -0.65 -0.26 0.22 2. -1.63 -0.95 -0. 88 -0.50 0.31 0.76 3. -1. 83 -1.83 -1.63 -0.82 -0.31 0.55 0.95 4. -1.27 -1.09 -0.50 0.04 0.88 1.37 5. -1. 63 -1. 27 -1. 02 -0.60 -0.04 . 0.26 0.73 1.09 6. -1. 83 -1.37 -1.09 -0.76 -0.22 0.26 1.17 7. -1. 83 -1.49 -0.76 -0.31 0.26 0.82 1.49 8. -1. 83 -1. 49 -1.49 -1.02 -0.65 -0.17 0.65 1.17 9. -1. 49 -1. 27 -1.02 -0. 70 -0.04 0.31 0.82 1.37 10. -1. 83 -1.83 -1.83 -1.27 -0.65 -0.04 0.59 11. -1. 83 -1. 63 -1.27 -1.27 -0.54 -0.17 0. 35 1.17 12. -1. 83 -1. 83 -1.49 -1.27 -0.95 -0.54 0.00 0.70 13. -1. 83 -1.49 -1.09 -0.55 -0.17 0.40 1.37 14. -1.63 -1.63 -1.17 -0.70 -0.13 0.88 15. -1. 63 -1. 09 -0.95 -0.70 -0.45 0. 00 0.45 1. 37 16. -1. 83 -1.49 -1.37 -1.17 -0.76 -0.09 0.71 17. -1.83 -1.49 -1.09 -0.88 -0.35 0.76 18. -1. 37 -1.09 -0.88 -0.65 -0.26 0.40 1.17 19. -1. 83 -1.63 -1.02 -0.40 0.17 0.70 1.48 20. -1.49 -0.95 -0.13 0.40 0.70 1.17 21. -1.83 -1.37 -0.88 -0.40 0.22 0.82 22. -1.48 -1.02 -0.65 -0.26 0.31 0.95 23. -0. 95 -0.76 -0.50 -0.17 0.50 1.09 1.83 24. -1. 83 -1. 37 -1.24 -0.50 -0.09 0.55 1.17 1.63 25. -1.48 -1.27 -0.76 -0.50 0.09 1.02 26. -1. 63 -1. 17 -1.02 -0.40 0.09 0.35 1.09 1.63 27. -1. 48 -1.09 -.088 -0.70 -0.31 0.45 1.17 28. -1.17 -0.65 0.13 0.76 29. -1. 63 -1.37 -1.09 -0.65 -0.13 0.60 1.17 30. -1. 83 -1.49 -1.17. -0.82 -0.65 0.17 0.95 31. -1.37 -1.09 -1.02 -0.76 -0.50 0.45 32. -1. 83 -1. 17 -0.95 -0.60 -0.09 0.13 0.76 1.49 33. -1. 83 -1. 63 -1.27 -0.82 -0.50 0.09 0.82 34. -1. 83 -1.83 -1.63 -1.02 -0.50 0.17 1.09 No unit normal deviates were calculated where the proportion of responses i n the category was less than 0.02. 137 STATEMENT CATEGORIES OF IMPORTANCE 1 2 3 4 5 6 7 8 9 35. -1.83-1.48 -1.27 -1.02 -0.65 -0.40 0.13 0.95 36. -1.27 -1.09 -0.45 0.45 37. -1.63 -1.48 -1.09 -0.95 -0.76 -0.31 0.55 1.37 38. -1.83 -1.63 -1.27 -1.17 -0.82 -0.70 -0.13 0.88 39. -1.63 -1.37-1.09 -0.95 -0.50 -0.26 0.50 1.27 40. -1.83 -1.83 -1.17 -0.95 -0.65 -0.22 0.70 1.17 41. -1.63 -0.82 -0.31 0.35 1.17 42. -1.63 -1.63 -1.48 -1.02 -0.60 -0.13 0.50 43. -1.83-1.48-1.09-0.88-0.17 0.45 44. -1.63 -1.37 -1.02 -0.65 -0.22 0.35 1.02 45. -1.27 -0.76-0.55 0.00 0.88 Those three values for each of the f o r t y - f i v e statements are provided i n Table F. The semi-interquartile range, l a b e l l e d "Q", was calculated as P_ -P„,_, and the median value of importance / 5 2 5 was P^„. Each of the values was calculated by determining where 50 the s p e c i f i c percentile was located on the o r i g i n a l frequency d i s t r i b u t i o n , and then locating the value of the percentile within the same category on the common continuum, but considering the new values on the continuum and the pa r t i c u l a r width of the category i n which the percentile was located. Testing the Index After the common continuum was constructed, an i n t e r n a l consistency test was performed to determine i f the values r e s u l t i n g from the t h e o r e t i c a l construction were consistent with the empirical data. The test was based on the degree of error present i n reconstructing the o r i g i n a l set of data, using only the information available within the common underlying 138 TABLE E INTERVALS BETWEEN UNIT NORMAL DEVIATES OF SUCCESSIVE CATEGORIES STATEMENT CATEGORY INTERVALS 2-1 3-2 4-3 5-4 6-5 7-6 8-7 1. 0.14 0.40 0.44 0.39 0.48 2. 0.68 0.07 0.38 0.81 0.45 3. 0.10 0.10 0.81 0.51 0.86 0.40 4. 0.18 0.59 0.54 0.84 0.49 5. 0.36 0.25 0.42 0.56 0.30 0.47 0.36 6. 0.46 0.28 0.33 0.54 0.48 0.91 7. 0.34 0.73 0.45 0.57 0.56 0.67 8. 0.34 0.24 0.23 0.37 0.48 0.82 0.52 9. 0.22 0.25 0.32 0.66 0. 35 0.51 0.55 10. 0.18 0.19 0.19 0.62 0.61 0.63 11. 0.20 0.36 0.36 0.37 0.37 0.52 0.82 12. 0.17 0.17 0.22 0.32 0.41 0.54 0.70 13. 0.34 0.40 0.54 0.38 0.57 0.97 14. 0.23 0.2 3 0.47 0.57 1.01 15. 0.54 0.14 0.25 0.25 0.45 0.45 0.92 16. 0.34 0.12 0.20 0.41 0.67 0.80 17. 0.34 0.40 0.21 0.53 1.11 18. 0.28 0.21 0.23 0.39 0.66 0.77 19. 0.20 0.61 0.62 0.57 0.53 0.78 20. 0.54 0.82 0.53 0.30 0.47 21. 0.46 0.49 0.48 0.62 0.60 22 . 0.46 0. 37 0.39 0.57 0.64 23. 0.19 0.26 0.33 0.67 0.59 0.74 24. 0.46 0.13 0.74 0.41 0.64 0.62 0.46 25. 0.21 0.51 0.26 0.59 0.93 26. 0.46 0.15 0.62 0.49 0.26 0.74 0.54 27. 0.39 0.21 0.18 0.39 0.76 0.72 28. 0.52 0.78 0.63 29. 0.26 0.28 0.44 0.52 0.73 0.57 30. 0.34 0.32 0.35 0.17 0.82 0.78 31. 0.28 0.07 0.26 0.26 0.95 32. 0.66 0.22 0.35 0.51 0.22 0.63 0.73 33. 0.20 0.36 0.45 0.32 0.59 0.73 34. 0.10 0.10 0.61 0.52 0.67 0.92 35. 0.35 0.21 0.25 0.37 0.25 0.53 0.82 Where i n t e r i o r categories have proportions of 0.0 the i n t e r v a l derived from the cumulative proportion has been divided equally among those i n t e r i o r categories. 139 STATEMENT CATEGORY INTERVALS 2-1 3-2 4-3 5-4 6-5 7-6 8-7 36. 0.18 0.64 0.90 37. 0.15 0.39 0.14 0.19 0.45 0.86 0.82 38. 0.20 0. 36 0.10 0.35 0.12 0.57 1.01 39. 0.26 0.28 0.14 0.45 0.24 0.76 0. 77 4.0. 0.33 0.33 0.22 0.30 0.43 0. 92 0.47 41. 0.81 0.51 0.66 0.82 42. 0.07 0.08 0.46 0.42 0.47 0.63 43. 0.35 0.39 0.21 0.71 0.62 44. 0.26 0.35 0.37 0.43 0. 57 0.67 45. 0.51 0.21 0.55 0. 88 Sum of 4.90 7.69 12 .92 17.69 18.26 28.04 31.43 Intervals Number of 15 30 41 43 45 45 44 Intervals Mean 0. 327 0.256 0.315 0.411 0.406 0.623 0.714 Interval Width Cumulative 0.327 0.583 0.898 1.309 1.715 2.338 3.052 Width scale and the t h e o r e t i c a l d i s t r i b u t i o n s accompanying i t . Edwards reported an average discrepancy i n the range of 2.4 per cent, and indicated that the empirical data was suited to the assumptions made for the construction of the continuum when the discrepancies were within that range. The average deviation found i n the present study was 3.0 per cent. I t was therefore concluded that the assumptions were acceptable and that the data was suited to the transformation procedure. Further tests were also conducted with respect to the Index of Program Planning, to estimate the possible e f f e c t s 140 TABLE F SEMI-INTERQUARTILE RANGE AND SCALE VALUES OF STATEMENTS STATEMENT P75 P2 5 Q P50 1. 3. 37 1.68 1.69 2 .87 2. 2.93 1.51 1.42 2 .10 3. 2 .59 . 1.41 1.18 1.94 4. 2.21 1.17 1.04 1.68 5. 2.25 0.83 1.42 1. 37 6. 2 .72 1. 36 1.36 2 . 00 7. 2.20 0.97 1.23 1.53 8. 2.36 1.28 1.08 1.85 9. 2 .18 0.91 1.27 1.36 10. 3.13 1.69 1.44 2 .39 11. 2.67 1.22 1.45 1.92 12. 3.03 1.57 1.46 2.34 13. 2.60 1.19 1.41 1.91 14. 2 .93 1.74 1.19 2 .44 15. 2 .57 0.94 1.63 1.72 16. 3.03 1.78 1.25 2.42 17. 3.01 1.93 1.08 2.58 18. 2.64 1.26 1.38 1.96 19. 2.31 1.10 1.21 1.59 20. 2.29 1.01 1.28 1.41 21. 2 .90 1.47 1.43 2 .12 22 . 2 .78 1.28 1.50 2.00 23. 1.93 0.68 1.25 1.42 24. 1.87 0.71 1.16 1.37 25. 2.84 1.43 1.41 2 .24 26. 2 .03 0.74 1.29 1.23 27. 2.61 1.34 1.27 1. 97 28. 2.97 1.69 1.28 2.23 29. 2.46 1.28 1.18 1.83 30. 2 .84 1.64 1.20 2.20 31. 3.25 1.90 1.35 2.71 32. 2.27 0.82 1.45 1.47 33. 2.23 1.07 1.16 1.65 34. 2.78 1.56 1.22 2 .17 35. 2 .85 1.28 1.57 2.18 141 STATEMENT P75 P2 5 Q P 5 0 36. 3.25 2.08 1.17 2 .69 37. 2 .49 1.38 1.11 1.94 38. 2.93 1.74 1.19 2.44 39. 2 .54 1.13 1.41 1.93 40. 2.32 1.27 1.05 1.87 41. 2 .67 1.41 1.26 2.01 42. 3.21 1.63 1.58 2.49 43. 3.25 1.87 1.38 2.54 44. 2 .72 1.28 1.44 1.95 45. 2.91 1.46 1.45 2.34 upon the scale had a d i f f e r e n t group of tr a i n i n g judges been used to develop the scale. For purposes of the tests, the group of f i f t y - e i g h t judges was randomly sorted into two separate groups, using the toss of a coin as the c r i t e r i a for group selection. As a r e s u l t of that procedure, the f i r s t random group consisted of twenty-three judges, and the second group included t h i r t y - f i v e judges. The two groups were then compared with respect to the median values of importance for the f o r t y - f i v e statements i n the t h i r d exercise, had each group been i n d i v i d u a l l y used i n the development of the instrument. The c o r r e l a t i o n between the two random groups was considered to be an estimate of the co r r e l a t i o n which might have existed between the actual group of judges and any other group of judges. The value of the co r r e l a t i o n was +0.84, and that s t a t i s t i c was s i g n i f i c a n t beyond the .001 l e v e l of signi f i c a n c e . I t was therefore concluded that another set of judges would l i k e l y have arrived at a similar set of median values for the f o r t y - f i v e statements. The two random groups were also compared with respect to 142 the degree of agreement exhibited toward the importance of the f o r t y - f i v e statements included i n the t h i r d exercise. The disagreement among judges i n the t o t a l group of f i f t y - e i g h t was evident, and the c o r r e l a t i o n between the two random groups was found to be +0.05. I t was therefore concluded that another set of judges might e a s i l y have selected another set of statements for the Index of Program Planning, as the s t a t i s t i c was not s i g n i f i c a n t even at the .25 l e v e l . Further testing was then conducted to determine which s p e c i f i c statements might have been selected for the instrument by each of the random groups. The f i r s t group would have selected twenty-four statements according to the c r i t e r i a which were used for selection of statements by the t o t a l group of judges, and the second random group would have selected twenty-seven statements. The two groups would have selected fourteen statements i n common, and each of those fourteen statements i s included i n the Index of Program Planning. A summary of the statement selections may be seen i n Table G. TABLE G COMPARISON OF STATEMENTS SELECTED BY TWO RANDOM GROUPS OF JUDGES STATEMENT RANDOM RANDOM INDEX OF GROUP 1 GROUP 2 PROGRAM PLANNING 1. 2. * 3. * * 4 m * * * 5. * •6. * 7 1 * * * 8. * * 143 STATEMENT RANDOM RANDOM INDEX GROUP 1 GROUP 2 PROGRAM 9. * * 10. * 11. * 12 . 13. * 14. * * * 15. 16. * * * 1.7. * * * 18. * 19. * * * 20. * * 21. * 22 . * 23. * * * 24. * * * 25. * 26. * * * 27. * * 28. * * 29. * * * 30. * * * 31. * * 32. * 33. * * 34. * * 35. * 36. * * * 37. * * * 38. * * 39. * 40. * * * 41. * * 42. 43. 44. 45. Indicates that the statement was selected for i n c l u s i o n i n the instrument by that p a r t i c u l a r group of judges. 144 APPENDIX D INDIVIDUAL AND CORPORATE PARTICIPANTS IN THE STUDY Ace, Dr. Merle Assistant Professor, Faculty of Commerce and Business Administration, U.B.C. Anders, Sergeant Erwin Management Development Coordinator, Cit y Police Department, Edmonton Baker, Dr. Harold Director, Extension Division, University of Saskatchewan, Saskatoon Bath, Bob Training Manager, Imperial O i l Ltd., Redwater, Alberta Beattie, Jack Coordinator - Marketing Training Western, Imperial O i l Ltd., Edmonton Blank, Dr. Stan Associate Professor, Faculty of Education, U.B.C. Bloomer, Newc ombe Personnel Development Advisor, Imperial O i l Ltd., Edmonton Brown, Pat Management Consultant, Kates Peat Marwick & Co., Vancouver Clarke, J.M.C. Director, Research, Workmen's Compensation Board, Vancouver Cowan, D.C. Head, Management Development Section, Manpower Training Branch, Department of Manpower and Immigration, Ottawa Cunningham, Miss R.J. Training Coordinator, Vancouver General Hospital Currie, R.J. Partner, P.S. Ross & Partners, Vancouver D'Amur, Bob Advisor - Training, Gulf O i l Ltd., Edmonton Refinery 145 Dart, Dr. Jack Associate Professor, College of Commerce, University of Saskatchewan, Saskatoon Day, William Dean of Continuing Education, Douglas College, New Westminster, B.C. Dickinson, Dr. Gary Assistant Professor, Adult Education, U.B.C. Doell, Jack Training Director, Potash Company of America, Saskatoon Dyke, Robin Consultant, P.W. Easton & Associates, Vancouver Easton, Pat President, P.W. Easton & Associates, Vancouver Edmonds, Ron Assistant Professor, College of Commerce, University of Saskatchewan, Saskatoon Everard, Derek President, Derek V. Everard Associates Ltd., Edmonton Ewing, Tom Personnel Director, Mannix Construction, Calgary F a r r e l l , Dr. Glen Head, Program Development Department, Extension Division, University of Saskatchewan, Saskatoon Faryon, Dick Associate Professor, Department of Extension, University of Alberta, Edmonton G i l l i e s , Don Training O f f i c e r , Mclntyre - Porcupine Mines, Grande Cache, Alberta Heath, Morris Director, Information Services, The Conference Board i n Canada, Ottawa Hegel, Ed Coordinator, Management Programs, Extension Division, University of Saskatchewan, Saskatoon 146 Henderson, Norm Program Consultant, Department of Manpower and Immigration, Vancouver Hiron, Don Director of Training and Development, C i t y of Edmonton Holfeld, Treavor Personnel Development Advisor, S h e r r i t t Gordon Mines Ltd. Fort Saskatchewan, Alberta Hume, David Coordinator, Industry Services, B.C.I.T., Burnaby, B.C. Hurka, Dr. Slavek Associate Professor, College of Commerce, University of Saskatchewan, Saskatoon Johanson, Gary Training Coordinator, Strathcona Refinery, Imperial O i l Ltd., Edmonton Kindred, Herb Director, Extension Department, University of Saskatchewan, Regina Campus King, Davis Director, Management and Productivity Center, B.C. Research, Vancouver Kobylka, Ken Management Development and Planning Coordinator, Parks and Recreation Department, City of Edmonton Lawson, Terry Consultant, P.W. Easton & Associates, Vancouver Lee, Alec Research Coordinator, Union of B.C. Indian Chiefs, Vancouver Linder, George Administrative O f f i c e r , D i v i s i o n of Continuing Education University of Calgary Lockhart, Fred Training O f f i c e r , Federated Cooperatives, Saskatoon Lockwood, Charles Assistant Director, Extension Department, University of Alberta, Edmonton 147 Michaels, Mrs. Dale Marketing Department, B.C.I.T., Burnaby, B.C. Middleton, Bob Regional A g r i c u l t u r a l Extension Coordinator, Saskatchewan Department of Agriculture, Regina M i t c h e l l , Dr. Vance Associate Professor, Faculty of Commerce and Business Administration, U.B.C. Moore, Dr. Larry Associate Professor, Faculty of Commerce and Business Administration, U.B.C. Morrison, Vic Staff Development O f f i c e r , Government of Alberta, Edmonton Naismith, E a r l Continuing Engineering Education, Douglas College, New Westminster, B.C. Niemi, Dr. John Associate Professor, Adult Education, U.B.C. Ondrack, Jack President, International Brick and T i l e , Edmonton Parker, George Program Consultant, Trade and Occupational Training, Department of Continuing Education, Regina Rubidge, Nick Graduate Student, Adult Education, U.B.C. Ruttle, Ken Secretary to the Board of City Commissioners, Calgary Scharf, Alan I n d u s t r i a l Engineer, I n d u s t r i a l Services Division, Saskatchewan Research Council, Saskatoon Sel l e r s , B i l l Director of Training and Development, Federated Cooperatives, Saskatoon Seminoff, Nik Media S p e c i a l i s t , Extension Division, University of Saskatchewan, Saskatoon 148 S i l v e r sides, Frank Professor, College of Commerce, University of Saskatchewan, Saskatoon S t i e f e l , Dr. Rolf Professor of Management Development, University of Geneva, Switzerland Taylor, Dr. Ron Assistant Professor, Faculty of Commerce and Business Administration, U.B.C. Thorn, Gordon , . . . V i c e - P r i n c i p a l , Extension, B.C.I.T., Burnaby, B.C. Thomas, Ed Organization Planning, Development, and Training Manager, Proctor and Gamble, Grande P r a i r i e , Alberta Thornton, Dr. James Assistant Professor, Adult Education, U.B.C. Trapp, George Manager of Training, Sask Power, Regina Tubb, Dick Supervisor, Business and I n d u s t r i a l Training, Department of Advanced Education, Government of Alberta, Edmonton Turner, Tom Consultant, P.W. Easton & Associates, Vancouver Verner, Dr. Coolie Professor, Adult Education, U.B.C. Virtue, Barry Provost, Mount Royal College, Calgary Wadsworth, Mrs. Pat Director, Ambulatory Care and Special Services, Vancouver General Hospital Whale, Dr. Brock Assistant Director, Extension Division, University of Saskatchewan, Saskatoon Wilson, Peter Associate Professor, Extension Division, University of Saskatchewan, Saskatoon 0 CORPORATIONS Ai r Canada Bank of Montreal B.C. Forest Products B.C. Hydro B.C. Packers B.C. Telephone Canadian P a c i f i c A i r l i n e s Canadian Cellulose Dominion Bridge Eatons of Canada Finning Tractor & Equipment Canada Safeway Macmillan and Bloedel Rayonier of Canada Royal Bank of Canada Shell O i l of Canada Simpson's Sears The Bay TPL Industries Tahsis Company Trans Mountain O i l Pipeline Weiser Lock Company Weldwood of Canada APPENDIX E INTERVIEW SCHEDULE COMPANY PERSON B)JTERVIEWED POSITION IN COMPANY Respondent Number Card Numbjr What is the general nature of the busiiess; in which this company is engaged? How many people are employed by the company? Is this company: 1. Privately Owned? 2. Publicly Owned? 3. A Crown Corporation? . U» A Cooperative? In which city i s the head office of th>3 company located? What was the total company payroll for the past fiscal year? $ What was the total gross sales for the company in the past fi s c a l year? $  Does your company have a formal policy relating to training matters? 1. Yes; 2# No 3. What percentage of management and supe:rvlsory training is conducted during company time? How many days per year of company time i s usually allowed for the training of: 1. an "average" manager 2. an "average" supervisor 3* an "average" non-supervisory worker What percentage of fees are normally reimbursed to employees upon successful completion of job-related courses offered through various educational agencies? What i s bop management's general attitude toward company training programs? 1. very favorable 2. somewiat favorable 3. non-committal 4» somewhat unfavorable 5. very unfavorable How willing i s the company to set aside funds for worthwhile company training programs? 1. very f i l i n g 2* moderately willing 3. generally unwilling The person i n your company who i s directly responsible for trailing operations i n British Columbia would be classed as: 1. top management 2. middlj management 3. lower management or supervisory 4. non-management or non-supervisory 5. no person is responsible for training How many training specialists are employed by the company on a full-time basis i n British Columbia? What percentage of the full-time training specialists i n British Columbia have been formally trained i n an educational field? (instructing, learning theory, etc.) What percentage of the full-time training specialists i n B r i t i sh Columbia have been transferred from another company department or function? What percentage of the full-time training specialists in British Columbia have been engaged in the company training function for less than two years? For how :>nany years have formal managenent and supervisory training programs been continuously offered by the company :ln British Columbia? What percentage of company managers and supervisors have participated in company training programs over the past 24 months in British Columbia? How many different training programs have been conducted for company managers and supervisors over the past 24 months i:a British Columbia? (Not including repetitions) 152 21. What was the company training budget amount for the past fiscal year in British Columbia? $ 58-63 22. What percentage of the company training budget in British Columbia is normally allocated for: 1. Direct Instruction % 64-65 2. Evaluation and Research % 66-67 3. Program Planning and Development % 68-69 4. Purchase of Equipment and Training Materials % 70-71 23. Does the company subscribe to either Training and  Development Journal or to Training i n Business and Industry? 1. Yes 2. No 3. Uncertain 72 Respondent Number 1-3 Card Number 2 4 According to the scale which is provided, please indicate the overall degree of activity and effort which is being conducted within the company with respect to the following four statements. 1 2 3 4 5 6 7 8 9 No Moderate High Effort Effort Effort 24* . The company attempts in some formal ma oner to coordinate.; a l l hiring, training, and promotions with studies; of future manpower requirements. __ 5 25. The company i s engaged in projects designed to study and speculate on future roles, and responsibilities of its employees. 6 26. The company establishes plans for 5 to 10 years i n the future and those plans are made known to employees who are responsible for planning training programs. 7 27. The company i s engaged in planning projects designed to speculate, study, or recommend potential organizational changes; or developments.' _______ 8 Considering a l l of the management and supervisory programs conducted by your company in British Columbia over the past; 24 months, please indicate the percentage of programs where the following activities occurred. 28. The work records of managers and super/isors or their subordinates were analyzed for such indicators as;waste materials, rejects, re-runs, safety, sales, or other measures of job performance for the purpose of determining i f training was necessary. % 9 153 29» The personnel records of managers and supervisors or their subordinates were analyzed-for such indicators as absenteeism, lateness, turnover, grievances, demotions, or requests for transfer for the purpose of determining i f training was 10 necessary. 30* The instructional situation was designed to simulate actual working conditions as much as possible so that behaviors learned in the program could easily be transferred to the work situation. % H 31 . The program instructor was active in helping to design tie sequence of instructional events. % 12-32. The program was designed specifically for a group of employees whose working situation and training needs were knora to the program designers. % 13; 3 3 . Lesson plans or meeting procedures were put through pilot tests and were revised wherever necessary before the program was formally begun. % 14 34« A series of cost/benefit studies of alternative approach?s to the training program was used to convince management of the efficient use of funds by the proposed training program. % 15 35* Short preview programs were provided for the approval of top management before the f u l l scale program was begun. % 16 3J6. Management demands for cost controls were partly met by calculating both a variable and a fixed cost component for the training program. % 17 3-7. A return on investment for the program was calculated, i n order to convince management that the program was of value to the company. % IS 38» To facilitate program approval, management was asked to . participate i n the development of the program. % 19 39« When specific resource people from within the company were requested for the training program by the instructor the company attempted to obtain them and make arrangements for their release from other duties. % 20 40 . The instructor was provided with sufficient secretarial assistance to meet his requirements.before, during, and after the instructional phase of the training program. % 21 154 4 1 . A l l necessary audio-visual equipment needed for the program was provided by the company either through rental or ownership. % 22 4 2 o The training program was conducted in an area which was specifically allocated to the program by the company, and in which no other company activities were allowed to intrude during the conduct of the program. % 23 43. At least one member of the planning group for the program assisted the instructor throughout the instructional phase in making necessary adjustments, in the training program. % 24 44« Participants were evaluated at several stages i n the program to determine their progress toward achieving course objectives. $ 25 4-5. The objectives of the program were used as a basis from which tc evaluate the learning and achievements of the participants. % 26 46 . Before and after measures were made among the participants and among a similar group of employees not being trained with respect to objectives of the program for the purpose of determining the effects of the program. J> 27 47» The on-the-job behavior of participants was observed or measured both before and after the program to evaluate the effects of the training program. % 28 48« The planning procedures which were used, to develop the program were evaluated to determine i f improvements could have been made in the planning processes. % , 29 49. Short refresher programs were provided for participants at intervals after the original program was completed. % 30 Total Scale Score 31-33 End of Interview 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.831.1-0055930/manifest

Comment

Related Items