Evaluation of entrepreneurial development coaching: changing the Teachers’ thinking and action on entrepreneurship

Abstract

Very often it has been reported that the greatest obstacle in Entrepreneurship Education are the teachers and their attitudes towards entrepreneurship. In the Entrepreneurial Development Coaching(EDC) program an aim has been to change participants’ thinking and pedagogical actions in entrepreneurship education. In this process the educators used the entrepreneurial learning method. Several researchers (e.g. Gibb 1993, Kyrö 1997; Kyrö & Ripatti 2006; Kyrö, Lehtonen & Ristimäki 2007) have stated that entrepreneurship education needs to be taught entrepreneurially. This, in turn, makes the education entrepreneurial. Haahti (1989) followed the historical text sources and found the first meaning of entrepreneurship education which was action. Teaching entrepreneurship by knowledge-based methods alone does not affect the character and the nature of entrepreneurship and may actually have negative consequences. Conversely teaching whatever subject entrepreneurially affects values, attitudes, action habits and thinking and these will promote entrepreneurship. In this paper we deal with the research question: Does the new entrepreneurial pedagogy model, EDC, affect teachers’ thinking and action on entrepreneurship? Liisa Remes was the main educator onthe program and Anne Gustafsson-Pesonen was the lead researcher and lead evaluator of the education process and EDC program. In this paper they will demonstrate the entrepreneurial learning method, the structure of the course and the impact of the entrepreneurial course on teachers’ thinking and action on entrepreneurship.

Key Words:

• entrepreneurship education
• teacher education
• evaluation of the program
• thinking and action on entrepreneurship
• tacit knowledge
• knowledge coding

The attitudes of teachers are often seen to be the biggest obstacle to the successful teaching of entrepreneurship and the realization of entrepreneurship education. On the other hand, it has also been noted that the teaching of entrepreneurship seems to require the using of a concept of learning that is congruent with the phenomenon of entrepreneurship as the learning methodology of the teaching of the subject matter. The term ‘entrepreneurial pedagogy’ is used for this process, because learning is based on the theoretical information gained from the phenomenon of entrepreneurship (Remes 2001, 2003). The conception of learning teaches people ways of thinking, looking and acting needed in entrepreneurship, in other words, the values and attitudes of entrepreneurship, in addition to the information content of the education.

The EDC program was carried from September 2008 to April 2011 in South Savo, Finland. The main actors of the program were the vocational colleges, Esedu and Sami and Aalto University in South Savo, Finland. There were two main aims of the project: firstly, in the long term, to increase students’ entrepreneurial readiness in vocational colleges in South Savo, Finland and also enhance their entrepreneurship and business knowhow; secondly, in the short term, to strengthen the teachers’ entrepreneurship educational skills at the same schools.

The target group of the project was young and mature students and the teachers of certain upper secondary vocational schools. The indirect target group was the management and other personnel of upper secondary schools and co-operation partners from working life (e.g. local entrepreneurs, enterprises, the representatives of labor market and administration and regional business mentors).

The research question for the evaluation research was: does the new entrepreneurial pedagogy model affect teachers’ thinking and action on entrepreneurship? The main idea of the evaluation was to ascertain how the program influenced a teacher's opinions, actions and thinking on entrepreneurship. The target group for the evaluation comprised the participants on the EDC program, as well the directors and rector of the vocational schools. This research was undertaken utilizing Kirkpatrick's (Kirkpatrick & Kirkpatrick 2006) four-level evaluation method.

In the paper there is described the EDC programs entrepreneurial learning method and Gustafsson-Pesonen's and her research group background theory in evaluation research and the results of the evaluation of the EDC program. The entrepreneurial pedagogy EDC program spoken in the paper is planned together Gustafsson-Pesonen and Remes and run by main educator Remes and with other educators, according to the entrepreneurial learning method described earlier in Remes doctoral theses (2003). The background theory in evaluation research and whole research process owns by research group whom was led by Gustafsson-Pesonen. The whole research report and all data is possible to read in Finnish (Gustafsson-Pesonen, Heimonen, Rautio & Kantanen, 2011).

The paper is composed as follows: the first section deals with the previous research made of entrepreneurial pedagogy and the second with entrepreneurial teacher education. The third section introduces the background to evaluation research, while the fourth section states the methodology of the research. Following these come the results and finally the conclusion.

Entrepreneurial pedagogy

Alain Gibb is a well-known researcher of entrepreneurship education. He has done research on entrepreneurship education since 1985. Gibb (1993) said entrepreneurship education and whole term entrepreneurship should not be in schools business studies but entrepreneurship should be the ability to operate confidently in situations of uncertainty. Very often entrepreneurial pedagogy is still seen as the part of business studies and teaching entrepreneurship often uses business terminology and methods. This is one significant problem when talking entrepreneurial pedagogy as, for example, Kajanto, Kyrö and Saarelainen (2001), Fayolle and Gailly (2008), p. 569–593) have been said.

Entrepreneurial pedagogy term began to entrepreneurship education research in the 1990s (Deakins & Freel 1998; Young & Sexton 1997). After that several researchers of entrepreneurship education has been studied the entrepreneurial pedagogy and its expression (Diensberg 2008; Fayolle 2007; Gibb 2005a; Harrison & Leitch 2005; Kyrö 2005; Politis 2005; Rae 2000; Rae 2004a, Rae 2004b, Rae & Carswell 2001). For example, Rae and Carswell (2001) have been said human being is the most important instrument when talking entrepreneurship. Diensberg (2008) argues that the growth of the individualism is the best way of entrepreneurial pedagogy. He has also said it is important forget the classroom teaching. Gibb (2005a) said that entrepreneurial pedagogy grows from the essence of entrepreneurship. He argues (2005b) that entrepreneurial pedagogy is more action learning than only listening. It is experimental learning, team work and learning by doing. Fayolle (2007, p. 60) agrees with Gibb, learning by doing is the best practice of entrepreneurship. Kyrö (2005, p. 25) argues that teachers should be able to offer for students an environment where they can try to feelings, see, communicate and learn how to organize things.

In Finland, entrepreneurship has been defined as an individual's capability to create ideas and make them work usefully and beneficially. Entrepreneurship education is education to support this process (cf. OPM 2009).

In the EDC program were used a model of entrepreneurial learning of Dr. Liisa Remes. Remes (2003) evinced in her doctoral theses a notion that a learner's idealization is one part of the entrepreneurial learning process. Later Remes (2003) conducted teaching experiments based on the same idea: Let the learner find out what needs to be done and decide how to achieve this and with whom. The teacher's responsibility is to monitor the learning process and help the learner if he/she so requests. However, this is the case with your own business. When you learn to think and act entrepreneurially, in future, you have better skills to run a business of your own or act in the environments that demand entrepreneurial attitudes towards action.

Regarding how to teach entrepreneurship many researchers ask for learning that is logical in entrepreneurship (e.g. Gibb 1993; Kyrö 1997; Kyrö & Ripatti 2006; Kyrö, Lehtonen & Ristimäki 2007). Remes (2001, 2003) researched learning paradigms and studied the history of education to ascertain if traditional learning and teaching supports learning to be an entrepreneur or to act in an entrepreneurial manner. Instead, so-called reform pedagogies do support it. The learning is then child or student centered and learning environments activate the learner holistically. Learning is also combined with the social authentic environment and is felt to be practical and actual. (Remes 2001, 2003)

Table 1 describes what happens in learning when the learner is at the center of his or her learning process and carries the process holistically from the very beginning (finding the problems to solve or projects to run) to the ultimate responsible product-making. In this model product is a very wide concept. It may be e.g. a book, services, math or drawings or a marketing project.

Table

Starting-point: Finding a problem to solve or project to run	Freedom creativity	Uniqueness risk-taking	Responsibility functionality holism	A holistic idea of man, world and epistemology
Individual learner	Self-direction	Recognizing one's own special character	Strong Self-esteem	Meaningful learning
Communality group	Target-oriented communication	Social skills different people	Environment-respecting activity	Skills needed in society
Materiality	Capability to create new products	Planning skills	Product-making skills	Life-management
Entrepreneurial method	Creation choices	Decisions evaluation	Realization production	Life-long learning

The Table 1 is viewed from left to right. It can be cross-tabulated and it can be thus read to find out what entrepreneurial pedagogy teaching does. The values of entrepreneurship are in the top row. The left column shows the levels of learning. The last row describes the learning process and the left column the total affection of each row.

In fact, in so-called traditional learning and teaching we separate the learner very effectively from finding the projects necessary to make or the problems to solve, flowing ideas, planning and risk-taking both responsibility–all necessary for entrepreneurship and making business. Remes claims that only product-making skills are active in traditional learning and teaching. This socializes effectively for passivity and un-entrepreneurial attitudes towards life. The values of entrepreneurship are creativity and freedom, uniqueness and variety and responsibility and comprehensiveness (top line).

The learning process of entrepreneurship includes the conceptualization, creation, choice-making and the responsible making of a problem or some other need for action (bottom line).

When cross-tabulating the values and learning process of entrepreneurship, it was observed that entrepreneurial pedagogy teaches the main things according to the nine squares in the middle of the table. These can be used as the basis of the qualitative evaluation of entrepreneurial learning.

The entrepreneurial pedagogy described above is the part of learning in which the learner applies knowhow in a genuine/authentic situation that calls for their entrepreneurship. In addition to this, learning includes the teaching of information with the so-called more traditional methods. The learner applies this knowledge in entrepreneurial pedagogy and learns also new things quite extensively. Entrepreneurial pedagogy could thus be described as learning in which the learner creates new information to suit the current needs. (Remes 2001, 2003)

Entrepreneurial development coaching

According to entrepreneurial pedagogy, the learning process of entrepreneurship is therefore a creative learning event, in which the learners quite independently create new products and services. The teacher's responsibility is to create the outward circumstances for learning, to support and instruct learning and to function as an entrepreneurship mentor.

The teacher's work as an entrepreneurship mentor requires that the teacher is familiar with the learning process of entrepreneurship. As was done in the project education, the teacher has to experience the process of entrepreneurship by her or himself to know empirically how to instruct the pupils and students. This is why entrepreneurial pedagogy should be used as the conception of learning in teacher education where entrepreneurship or entrepreneurship education is studied; in other words, the method should be entrepreneurship in the education.

Figure 1 shows the structure of the entrepreneurial development coaching, EDC course whom is evaluated in this research.

Fig. 1. The structure of the EDC course planned by Gustafsson-Pesonen and Remes

The aim of the EDC program was that entrepreneurship would be assumed as one's own method of teaching. The contact teaching, workshops, and the implementation of the development project corresponded to the level of communality and materiality supporting the learning of the individual. The education included contact teaching of other instructors, too. Contact teaching included entrepreneurship and knowledge about entrepreneurship education that was later applied in the workshop. Fig. 2 presents types of knowledge that are activated in an entrepreneurial learning process.

Fig. 2. The process of entrepreneurial pedagogy by Remes.

In entrepreneurship existing knowledge is applied to the activity. New knowledge is acquired as needed, and it is also applied to the ongoing process. Entrepreneurial learning process needs knowledge that the actor is seen to have already but this is supplemented according to the need in the course of the process. The model imitates entrepreneurship that has the accomplishing of a product or a service for someone at the center of the activity. The activity is entrepreneurial and other people are in a significant role in it.

The nature of knowledge of entrepreneurship is present. Thus the task of the main instructor is to support learning in real time. When questions or a need for additional information arise in the process, the instructor has to be easily reachable. The main instructor serves as a role model for the developing entrepreneurship educator model of those having participated in the training.

According to Fig. 3, factual knowledge is taught through traditional lecturing methods, although the studying is nevertheless quite dialogical. In a workshop the role of the teacher is more instructive and the student is expected to personalize the previous knowledge. In the development project, or in the innovation of a pedagogic product, the role of the teacher is in accordance with entrepreneurial pedagogy, and the learner is helped only on request. Like a coach, the instructor supports and motivates in the creating of new things. The general goal of the process is to develop as an entrepreneurship educator.

Fig. 3. Levels of entrepreneurial knowhow, Remes

Fig. 3 presents the levels of knowhow in teacher education. These are individual entrepreneurship, a social dimension activating internal entrepreneurship and the level of implementing the development project or a pedagogic product. When the values of entrepreneurship color all the levels, the studying is different from what the teachers may have been used to. A learning process in accordance with entrepreneurial pedagogy also entails taking personal responsibility for one's own learning. The process tried to promote interaction and the composition of groups was free. Development tasks could be done alone, in pairs or in groups, because we wanted to give the trainees as much freedom to make choices as possible.

Background to evaluation research

The evaluation of the project EDC program was the evaluation of the impact of the operations of the program in a teacher's work and more extensively later on the organizational level work.

When discussion effectiveness evaluation research it could be said it is that it is quite difficult to measure what effects, how and to whom.

Like Fayolle (2005) has been studied the effectiveness of the programs and he has been said:

While the impact of the program or session on the trainees, students and other participants may be an acceptable evaluation criterion, the problem of how to measure it still remains. What indicators should be used, and how should they be measured? How can you measure a change in someone's state of mind or behavior? How can the importance of the time factor be taken into account? And how can factors relating to education, teaching and training are separated from all the other factors that have an impact on the decision to choose a specific career path or profession?(Fayolle, 2005).

According to researchers, evaluation means the quantification of the value of phenomena and activity, based on the publicly expressed criteria, its proportioning to the evaluation base (for example, House 1993, p. 4; House & Howe 1999). As Raivola (2000a, p. 65–66) states: ‘the function of evaluation is to yield systematic information, which value- and advantage implications are attached to, so that information can be used to direct some social activity to reach the goals set for it even better and more efficiently than before.’ The utilization of evaluation information is a part of the effectiveness of the evaluation of education. In education effectiveness generally means the positive success of the education, pursued by action, and the fulfillment of its objectives and functions (Raivola 2000b, p. 12).

According to a division established in evaluation research, the utilization of evaluation information can be instrumental, conceptual or persuasive (Lampinen 1992, p. 30–37; Leviton & Hughes 1981, p. 528–548; Rossi, Lipsey & Freeman 2004). According to Chelimsky (1997, p. 10–18), we can pursue three ends with evaluation: fulfilling accountability, yielding new information or promoting development.

Evaluation and its effects can also be examined on different dimensions of time, according to whether it concentrates on evaluation during the activity (ex nunc), before it (exante) or after it (ex post) (Evaluating EU Expenditure Programmes 1997). In the analysis of the article in question and the data of the study related to it, information on the project before, during and after will be partly yielded.

In terms of time, the evaluation of different development projects can be carried out in several different stages of the project being evaluated. Traditionally ex post evaluation has been extensively utilized in the evaluation of different development projects, in which case the evaluation has often been concentrated on the effectiveness and implications of the outcome of the development projects. The content themes of evaluation in ex post evaluation are significantly related to the reaching of the set goals, change comparisons, viewpoints on learning and economic and functional things. (Anttila 2006, 2007)

It has been noticed that real-time evaluation of development projects has become common. In real-time evaluation operations taking place during the project are often tried to be clarified and the short-term results of the operations and the effectiveness on the operational processes and the actors are tried to be clarified in real-time. Additionally, ex post evaluation can be more and more utilized in respect to projects, for example so that the preliminary needs and views of the actors under operations are tried to be found out. On the basis of actor feedback, the content of the operations to be done in the project can be planned and guided in real-time, tailored to the actors (Lindström, 1994).

Vaherva (1983) considers the effectiveness of education to mean the achievement of the investment, process and output objectives set in detail for education in a certain, defined time span (long and/or short span). This assumes that the effectiveness of education can take place on different levels, for example, the readiness achieved by education can strengthen the mental and functional growth of the individual and more extensively, effectiveness can be manifested at the organizational level as changes in productivity, excellence and networking, and extensively in society as the development of working life and well-being.

When the frame of reference of effectiveness evaluation is examined, the four-step model of the effectiveness evaluation of education, developed by Kirkpatrick (Kirkpatrick & Kirkpatrick 2006), is often mentioned. On the basis of Kirkpatrick's original model (Kirkpatrick & Kirkpatrick 2006), numerous similar evaluation models of the effectiveness of education have been created. They, however, mainly include the central elements of Kirkpatrick's classification. Kirkpatrick's (Kirkpatrick & Kirkpatrick 2006) effectiveness levels of education can be divided into the following levels:

1. Level of reaction: In the evaluation of the level of reactions the satisfaction of the trainees in the education in relation to their earlier opinions and experiences is ascertained. On the other hand, even positive satisfactory results do not necessarily guarantee the learning of the participants in general or changes in their thinking and actions

2. Level of learning: With the evaluation of the level of learning the tacit knowledge, skills and attitudes learnt by the trainees are ascertained. Learning can be measured in different ways, such as written feedback (evaluations), tests, skills assessments, portfolios and different preliminary and final level measures/enquiries, learning diaries and reflection discussions.

3. Level of behavior: The evaluation of the level of behavior tries to generate information on the application of the things learnt in the education/on the course into the individual's own work and /or functional behavior. Functional and behavior-related changes are often connected to the participants’ prior knowledge, knowhow, attitudes and motivation and the ability to apply the new, learnt things.

4. Effects of the change of behavior on the organization: On the organizational level we are often interested in what effects education has had on the activity of the whole organization. In organizations, education is often seen as an investment, for which some return is expected, that is, benefit for the whole organization in addition to the individual.

It has been decided the evaluation on project and in EDC program have to improve the real changes both participants thinking and action and also the changes on organization level on the long term. For that reason the effectiveness evaluation in EDC program was used the Kirkpatrick's (Kirkpatrick & Kirkpatrick 2006) four level evaluation. According to the view of the researcher, the data was collected and analyzed using Kirkpatrick's methods levels 1, 2 and 3. The fourth level is possible just three to five years after the program because analysis of the actual impact of the program on the community/organizational behavior is also needed. The real changes on organizational level need several years.

Methodology

The evaluation data was collected in three different stages (Kirkpatrick & Kirkpatrick 2006). The clarification utilizes Real-time (reaction), Subsequent (learning), and Long-term (behavior) evaluation information to evaluate the impact of the EDC program. The frame of reference of the analysis of the central information on advance evaluation is based on the mapping of the initial stage of entrepreneurship and entrepreneurship education of the personnel of the educational institutions participating in the EDC program. The mapping was carried out on the project at the beginning of the EDC program. Real-time evaluation is based on the feedback on the training, collected during the project, especially on the viewpoints of the participants on the content and importance of the training in terms of all 15 educational periods arranged. Subsequent evaluation is based on essay-type texts collected immediately after the training, in which the change in relation to the initial situation was examined. Longer-term examination is based on the evaluation of the situation ca. six months after the training was over. In the case of long-term evaluation the data was collected utilizing the method of theme interview.

Real-time evaluation of the training was done by collecting written feedback on the contact teaching days and workshops of the training after each training session. Fifteen of these contact teaching days and workshops were arranged on the project within the EDC program during the years 2009–2010. In the training feedback the participants evaluated the success of the training from the point of view of the actors who participated in the training, especially from the perspectives of the content and applicability of the training, the teaching methods used, the know-how and proficiency of the instructor and general satisfaction with the training. In terms of the measures (training and workshops) of the project collecting actor feedback was easier and continuous because giving feedback was part of the training and workshop days.

Subsequent evaluation was done immediately after the whole EDC program had ended. The participants were given a task of writing an essay-type text, as also in the initial evaluation, on how the training had affected the way they saw and experienced entrepreneurship education, business know-how, entrepreneurial pedagogy and entrepreneurial behavior in their own work. Eight essay-type texts were returned (N40), thus the response rate fell far behind that of the initial mapping stage. The most probable reason for this was that after a long training period the people may have been rather tired of writing more feedback on the impact of the training.

Long-term examination is based on the evaluation of ca. six months after the training ended. The data were collected utilizing the method of theme interview for long-term evaluation. Ten people (participants, team leaders, managers, director) were asked to attend the interview and they were interviewed personally, face-to-face.

Results

In this section researcher is presenting the results of the study. Briefly, it could be said the results are quite good and affect. The results are shown three levels using the Kirkpatrick's (Kirkpatrick & Kirkpatrick 2006) evaluation model: Reaction, Learning and Behavior. It is possible to present the fourth level after the several years. The changes, which took place in the EDC program allows teachers in thinking and behavior, are really significant.

Reaction: level 1

When using Kirkpatrick's evaluation method the first step is just after the program or training days. There were 15 training days during the program. In terms of the training days and workshops, feedback during the training was collected with a special assessment form. In this connection the success of the training was assessed based on the numeral and written feedback given by the people participating in the training. Total amount of participants were 40 people from vocational schools Southern Savo and these were answered the reaction level evaluation.

Numerical feedback

In the numerical feedback the participants were asked to assess the success of the training days from the point of view of the subject matter dealt with on the training periods, the applicability to practice, the methods and ways of presentation used by the instructors, the know-how and proficiency of the instructors and satisfaction with the training as a whole. These items the participants were asked to assess on a Likert-type scale 1-5, in which the extremes of the scale were the following: 1=useless and 5=useful in terms of the importance of the subject matter of the training periods. For the other themes the extremes of the scale were 1=passable and =excellent.

In light of the participant feedback received for the training days, it can generally be stated that the EDC (entrepreneurship developer coaching) of the project succeeded partly excellently and overall theme-specifically very well or well from the point of view of the participants using the feedback average of the training as the grounds. On the other hand, it is difficult to make a more specific analysis of the success of the training based solely on theme-specific averages. That's why it is important to analyze the written feedback.

Written feedback

Along with the numeral feedback for the training days, the participants were asked for written feedback giving their reasons for the numeral feedback, to note their own wishes for the subject matter of the training days and to make suggestions for improvement or development or other important feedback on the training from the point of view of the respondent.

In light of the analysis of the content of the written feedback we can distinguish four development themes (see Fig. 4). It is good to recognize these and to consider them in terms of what could be done to develop similar training periods in the future.

Fig. 4. Chances at level 1: reaction.

Generally assessed, the heterogeneity of the respondents was emphasized in the different needs related to the training. Some of the respondents would have wished even more extensive knowledge in the training and/or some wished an even stronger application of the knowledge to practice. Additionally, different time resources for the internalization of the knowledge and the development of one's own thinking and more functionality were called for in the training days. On the other hand, specific individual needs were also emphasized, such as the more profound analysis of different tests, the use of methods suitable for young people and the need for learning how to make a business plan. In terms of the assessment during the training, in the end the respondents had a chance to write in their own work if they had any other comments or ideas for development related to the training days, the learning process or the learning environments. On the basis of the analysis of the content of the received feedback, in this connection we can thematically mention the various challenges that emerged during the training, the implications of learning and the effects of the training on an individual level and the individual factors behind the implications and effects of learning. Fig. 4 presents the themes created from the assessment feedback with the help of content analysis and theme-specific typical training feedback.

Learning: level 2

The short-term impact of the EDC program was studied by sending all those consenting a short e-mail inquiry to which they were to answer with an essay. Altogether eight people of 40 answered the inquiry, and a summary of the findings describing changes in thoughts or activity from those essays was made.

When participants were asked about how their conceptions, thinking, pedagogic solutions or activity changed, the following four themes (see Fig. 5), emerged along with the training.

Fig. 5. Changes at level 2: learning.

Behavior: level 3

Long-term impact assessment was once again taken to a deeper level. The long-term assessment was done by using a method based on the theme interviews. A theme interview is well-suited as a data collection method in this situation because this way the level of research changes all the time to become more profound. It is to be noted that care must be taken not to let the researcher's own attitude, conceptions or thoughts direct the course of the research and affect the interviewee.Altogether 10 interviews were conducted. Respondents were both participants of EDC coaching, team leaders (managers) and director of the school.

Especially answers describing the change in the activity and thoughts were classified from the answers on the main themes and grouped into entities describing the wider direction of change in the community and the behavior of the teacher/top management. Both teachers having participated in the coaching and their superiors and the top management of the school participated in the interview. From the data it was able to rise up five main themes concerning changes in behavior. These are shown in Fig. 6 and see also the deeper analyzing in appendix 1. The Teachers have access to a new innovative entrepreneurial pedagogy on their subject. They have understood the teacher role has been substantially chanced, it is more coaching than teaching. The teachers want to do more team work than before the EDC program. Their attitude towards entrepreneurship is much more positive. From the organization and the managers the teachers are expected resources for the entrepreneurial pedagogy and they have a feeling the support has been got.

Fig. 6. Changes at level 3: behavior.

Conclusions

It was rather clearly seen that when entrepreneurial pedagogy is used in teacher education the thinking and action of the teachers changed. After the evaluation their mindset seems to be more entrepreneurial as it was before the coaching project. Therefore, we suggest that there should be more impact on what pedagogy is used in entrepreneurship education, also when training the teachers under the topic.

The evaluation of the overall affects of the clarification is based on the evaluation of the change in the thoughts, actions and behavior of the people participating in the training, according to Kirkpatrick's classification. In the evaluation study, using Kirkpatrick's (Kirkpatrick & Kirkpatrick 2006) method, it was possible to use 1, 2 and 3 levels but not the fourth level. Reason for that was that there was too short time after the EDC program. It is possible to evaluate the fourth level changes: organization level just three to five years after the program.

The main needs and challenges of the development of the training possibly affecting the satisfaction/dissatisfaction with the training mentioned by the respondents were classified in the following conclusions: the training created an increased need for information, some of the respondents wished for a stronger application of the knowledge gained in the training in practice, time resources for the internalization of the know-how and the development of one's own thinking are needed from the Head of the school, even more functional content in the training sessions and more sharing of the good practices and examples were requested.

According to the training assessment, it would seem that according to the feedback on the training the EDC program succeeded in creating strong prerequisites for learning and positive learning experiences for the learners by finding good instructors using teaching methods supporting learning and able to create an interactive atmosphere in the training.

When evaluating the impact of the training on the level of learning, four main themes that can be seen to have changed in the course of the training emerge from the data. The themes were grouped as a change in 1) pedagogy, 2) thinking, 3) action and 4) attitude. It can be said that in the course of the training and partly due to it the changes were mainly positive. New or developed pedagogic experiments had been made, thinking and attitudes towards entrepreneurship had changed in a more positive direction and an approach clearly supporting entrepreneurship had emerged in the activity.

The evaluation on level 3, behavior, was already on a deeper level of interpretation. At this stage, too, the data was classified into main themes with the help of which we tried to discern and find the dimensions of the impressiveness of the training. There were five main themes: 1) expectations of the organization, 2) team work and co-operation, 3) change in teachers’ thinking and role, 4) the new entrepreneurial pedagogy and 5) teachers’ entrepreneurial attitude. All themes proved the EDC program had succeeded well and the changes were quite positive except for theme 5. The teacher's entrepreneurial attitude should still change and there is quite lot of work to be done.

It can be stated that the findings yield a good basis for developing entrepreneurship. There is still a lot to be done, that is, the change has begun and now we need and organizational development approach, for which the finding of resources is also essential. The speed of the change in the role of behavior, thinking and the teacher is well under way. The expressions of entrepreneurial pedagogy, a new kind of learning culture and enterprise-school co-operation has started. Development is still needed in the changing of attitudes.

For the future research according to Kirkpatrick's four step evaluation method, it is possible to see how the coaching operates in the community, organizational level. But it will only be able to measure the level 4: Changes on behavior on the organization level when at least three–five years after the coaching ended, preferably more. The perception of organizational level –changes will only be possible when all the levels of the organization have been gone through with the theme of entrepreneurship and the issue has been taken from the individual level to the structures. Our recommendation is we have to do deeper theme interview so that the real changes in organizational level is possible to find out.

It must say, in this paper was the weakness of some kind of narrowness of the data. The first level the data was reasonably large (N=40). First, the total amount of the teachers was 300 in whole organization and second, every teacher participating in EDC program answered for the survey. But the other levels (2 and 3) the data should have been wider than it was. Therefore, the fourth level of data acquisition must be particularly accurate, so that a profound change in the organizational level is clearly demonstrated.

Our research calls after remarkable changes at the educational policy. When we want to support entrepreneurship in the society we should organize the teaching so that it affects on learners’ entrepreneurial attitudes and action habits. This is a question of how to teach. Speaking of entrepreneurship is not enough. Moreover, generally used pedagogical solutions are pretty much opposite to the entrepreneurial pedagogy which actually means that learners are socialized to act unentrepreuneurially. To be honest, we don't support entrepreneurship but instead we don't let it live at all at our schools which means that effective entrepreneurship education doesn't really happen there. Thus, teacher education should include to its policies entrepreneurial pedagogy and at the same time we should take care that entrepreneurial pedagogy is present at schools’ everyday-life. This means that entrepreneurship education should take seriously and it should be given a special attention at every level of the school system.

Appendix A: Changes at third level: behavior

• Expectation of Organization: The support of managers is really important if entrepreneurship is to be applied in every subject: there is a need for resources, time, coaching and strategic guidance.

• Team work and co-operation: Respondents said they needed more team work if they really wanted to add entrepreneurial methods to their work. Some new co-operation models have been initiated: team have developed an inner forum meeting monthly, but they said this is not enough

• The change of teacher's thinking and role: The teachers said, for example: ‘It is not the purpose that all our students become entrepreneurs but it is already a value in itself that they get a certain kind of readiness to understand an entrepreneurial attitude towards work. It was been highlighted that the teacher is a creator of opportunities, not the giver of ready models, students are empowered in their own decisions and teachers should be the backup. Entrepreneurship is becoming inspired, sticking one's neck out, life-management and self-direction.’ It was also said several times that ‘teachers should develop a strong self-respect, in that case the student, too, may venture to ask unexpected “entrepreneurial” questions.’

• The new entrepreneurial pedagogy: Respondents said they had included new entrepreneurial pedagogy to their teaching methods: I. ‘Problem-solving and team work adapted to teaching’, II. ‘Peer evaluation is used along with self-evaluation in evaluation, III. Young entrepreneurship model and 24 h entrepreneurship camps’, IV. ‘A new learning environment in which it is possible, for example, to offer foot massage for clients for a charge outside of school hours’, V. ‘The number of enterprises has subsequently increased’, VI. ‘It starts with personalizing the study curriculum so the students themselves can set the goals: what I want to do when I grow up and how I can affect the future myself, students must be responsible for their own learning’, VII. ‘Different kinds of projects have come to stay in teaching’, and VIII. ‘School – enterprise co-operation has REALLY been started (an enterprise representative, godparent enterprises have been created), Teachers’ periods of in-service training? have been invested in, for example a metal work teacher goes to learn in a metal company.’

• The Teachers’ entrepreneurial attitude: It is very understandable that teachers whose thinking on entrepreneurship is somehow negative should join the entrepreneurship program. I. ‘Often those who already have a positive attitude towards entrepreneurship go to teachers’ entrepreneurship coaching, not the others…’ Luckily it was said during the interviews: II. ‘The attitude of the teaching personnel towards entrepreneurship has become more favorable because nothing happens if the attitudes do not change.’ Respondents also highlighted that very many teacher still negatively disposed towards entrepreneurship think that it is something very difficult: III. ‘Resistance to change is apparent, new things are experienced as a threat because they are not known and people are wondering how much they will increase the amount of work. They do not necessarily see that the new will bring new opportunities.’ After the program respondents also unfortunately said: IV. ‘It is still often thought that people are born to be entrepreneurs and that is the reason why many (teachers) do not want to be involved in the developing of entrepreneurship on their own subject and also I very well understand people who never want to become entrepreneurs, it is very different to act in an entrepreneurial way than to found an enterprise of one's own.’