CORPORATE VENTURING EVALUATION: HOW START-UP PERFORMANCE IS MEASURED IN CORPORATE VENTURING DURING THE COLLABORATION PHASE

In the context of business performance assessment, current research often focuses on evaluating effects of innovation projects with start-ups. Subsumed as corporate venturing (CV), investigating the impact of these projects is becoming increasingly important. Thus, the number of studies on the ex-post value of corporate venturing projects (CVP) has steadily increased over the last years. In contrast, this research attempts to cover the evaluation of CVP in the period between from the contract conclusion to termination. By conducting interviews, this research primarily aims at identifying indicators applied for evaluating CVP performance during the cooperation of subjects in the automotive industry. The results obtained show that evaluation is mostly based on subjective criteria and that no formal indicators are implemented to manage CVP during the collaboration phase. This research contributes to literature by revealing practically applied CVP performance indicators during the collaboration phase and by discussing their weaknesses.


INTRODUCTION AND OBJECTIVE
Corporate venturing (CV) is the umbrella term for entrepreneurial activities regarding collaborations of established organizations with new and innovative businesses (Reimsbach and Hauschild, 2012). Even though the relevance of CV has been acknowledged in literature and in practice, the theoretical understanding is still in its beginnings (Maula et al., 2009). CV has been discussed in literature every now and then. Three major peaks of attention can be identified so far. In 1960 there was the first peak, when over 25% of the Fortune 500 businesses engaged into CV (Baldi et al., 2015). In 1980, the second attention peak was reached, when corporates strived for diversification. The third peak emerged in the 1990s. New technological trends, market opportunities and an improved legal environment with less regulation and tax incentives fuelled the engagement in CV (Dauderstädt, 2013). Today, the attention on CV is increasing again. This is based on the fact, that corporates more and more collaborate with other organizations in order to keep up with a disruptive and fast developing world (Pekkola and Ukko, 2016;Ferreira et al., 2012). CV is a mean for corporates to achieve their goals in order to fulfil stakeholders' needs. To achieve these goals, the management of CV becomes crucial. Thus, an encompassing understanding of how to manage CV, including the underlying mechanisms and indicators is needed.
Yet, to the knowledge of the authors, research on CV management during the collaboration phase is rare. Existing literature investigates the ex-ante and ex-post phases of CV, e.g. startup selection criteria or measurement of ex-post CV success. Even performance measurement literature does not offer a framework to evaluate CV during the collaboration phase (Pekkola and Ukko, 2016;Westphal et al., 2010). Yet, as stated by Busi and Bititci, a deeper understanding of CV and its evaluation measures is needed (Busi and Bititci, 2006). Also, from an empirical perspective CV proved its importance. An Accenture study reveals that 50 out of the top 100 companies from the Fortune 500 ranking engage in CV (Berthon et al., 2015). Simultaneously, the investment sum increased (Brigl et al., 2016). A Boston Consulting Group study states that German corporates invested 1,3 Billion US Dollar in CV over a five-year period from until 2016(Boston Consulting Group, 2016. Considering, that at least 50% of corporate venturing projects (CVP) fail, indicators to evaluate when to end CVPs would increase the economic performance of corporates {Carneiro 2012 #72D: 995}. Consequently, such indicators could not only be used to manage the relationship, but also to influence the accomplishment of CV goals.
Following the need for such research (Pekkola and Ukko, 2016;Bititci et al., 2012;Benson and Ziedonis, 2009), the present study aims to answer the following research question: which indicators are practically applied within the automotive industry to evaluate CVP during the collaboration of subjects? Consequently, the research goal is to identify indicators for the evaluation of CVP during the cooperation of subjects. Special emphasis will be held on indicators which signal CVP problems and let assume, that the termination of the cooperation is more economically advantageous for the corporate. Research will be carried out only in organizations working within the automotive industry. By answering the research question this research contributes to literature by offering first empirical insights into CVP performance evaluation during the cooperation phase and by revealing practically applied indicators in the automotive industry.
In the following, basic theoretical information and applied methods are made transparent. Then empirical results and most important insights are shared. Afterwards the results will be discussed, before a conclusion regarding the indicators used for CVP evaluation during the collaboration phase is drawn.

THEORETICAL FRAMEWORK
After having shown the theoretical and practical importance of CV, a common understanding of CV and its goals is created. According to literature, CV is the umbrella term for entrepreneurial activities regarding collaborations of established companies with new and inno-vative businesses (Chesbrough, 2002;Keil et al., 2008;Dushnitsky and Lenox, 2006;Freese, 2006). Alongside others collaboration forms, like alliances and joint ventures, CV focuses on establishing mutual beneficial relationships (Pekkola and Ukko, 2016;Lee and Kang, 2015). CV collaborations represents the closest form of relationships (Parung and Bititci, 2006). CV can be structured in intern and external CV. Internal CV refers to internal entrepreneurial activities. These are anchored by establishing a start-up inside the corporate's boarder. In contrast, external CV focuses on entrepreneurial activities outside the corporate's border (Dauderstädt, 2013). In more detail, external CV describes the idea that a large, well-established organization (the corporate) engages into a collaboration with a small autonomous business (the start-up) with high potential for growth and innovation (Reimsbach and Hauschild, 2012).
CV represents a financial investment. Driven by the desire for control and mitigation of risk, organizations either invest directly or indirectly into start-ups. When organizations focus on controlling a start-up, they normally take equity. When corporates strive for few liability and management complexity rather than control, they tend to take no equity (Neumann et al., 2019).
Corporates engage in CV based on the economical decision to jointly solve problems which cannot be solved by the corporate alone (Camarinha-Matos et al., 2009). Especially in the case of innovation start-ups are great CVP partners, since start-ups are more innovative than corporates and therefore offer access to innovation and new technologies (Chemmanur et al., 2014;Engel, 2011;Bititci et al., 2012). However, also start-ups benefit from CV. Mostly, resources and managerial support is provided by corporates. Financing also plays an important role. Based on this support, CV backed start-ups reach a higher output in innovation (Chemmanur et al., 2014). Consequently, CV serves to leverage the benefits for both organizations (Maula, 2001;Faisst, 2005;Dushnitsky and Lenox, 2006).
When engaging in CV, corporates pursue an individual set of strategic and financial goals (Reimsbach and Hauschild, 2012). Yet, corporates do not focus mainly on financial benefits, but rather on strategic ones (Covin and Miles, 2007;Dushnitsky and Lenox, 2006;Chesbrough, 2002). This can be seen in Tab. 1, reviewing CV goal definitions.
Even though CV goals are formulated differently, they can be categorized into 6 categories. The first category represents the goal to create innovation and gain insight into new technologies. The goal to enter and develop markets embodies the second category. Thirdly, the goal to create spill-over effects and crossselling opportunities builds a category. A fourth category includes the goal to positively influence the brand and an organization's reputation. The fifth category represents a more qualitative goal, namely the enhancement of (entrepreneurial) culture. Finally, the sixth category encompasses the goal to generate additional financial returns.
Literature argues, that although an organization might target different goals, the "window on technology" or the creation of innovation is the most important one (Baldi et al., 2015;Chemmanur et al., 2014;Reimsbach and Hauschild, 2012;Faisst, 2005;Dushnitsky and Lenox, 2006).
Obviously, a mechanism to manage the organization towards achieving its objectives and to evaluate the degree of objective realisation is needed (Pekkola and Ukko, 2016). This is known as performance measurement (PM). PM represents a systematic approach to plan, measure, monitor, assess, reward, and control the performance of organizations whilst using suitable methods and tools (Pekkola and Ukko, 2016;Kaack, 2012). In general, PM represents a learning system, which is constantly optimized and refined to enhance its information and steering function. More specifically, PM is a method to plan and conduct data collection regarding goal achievement. According to Westphal et al. PM comprises two main elements. The first element encompasses the definition of performance measurement systems (PMS) to describe how performance measurement is set-up and conducted. The second element represents the definition of dimensions and key performance indicators (KPI) to evaluate the business performance (Westphal et al., 2010).
The PMS's focus changed over time. Originally, formal and financially measurable infor-mation was collected to support management decision making. Today also external information concerning customers, markets, competitors and especially non-financial information plays an important role (Collier, 2005). Solely using financial measures is discussed critically, because strategic goals and effects get lost (Pekkola and Ukko, 2016). These effects and the status of goal realization can be transferred into indicators (Ferreira et al., 2012). These indicators refer to different dimensions, e.g. efficiency and effective-ness, internal and external or financial and non-financial measures (Pekkola and Ukko, 2016;Ferreira et al., 2012). Literature discusses the value of both, financial and non-financial KPIs.
The impact on business performance through PMS and KPIs is widely acknowledged (Pekkola and Ukko, 2016;Dauderstädt, 2013). Even though the establishment of a PMS for CVP evaluation during the collaboration phase seems to be worthwhile, research regarding this topic is extremely limited (Basu et al., 2011;Banik, 2011;Westphal et al., 2010;Weber, 2009). Systematically structured operational indicators, which evaluate the strategic outcome and CVP success do not exist (Dauderstädt, 2013;Kollmann and Kuckertz, 2010;Faisst, 2005). Consequently, empirical research reading applied indicators for CVP evaluation during the collaboration phase plays an important role in gaining in-depth understanding of CV.

SEMI-STRUCTURED INTERVIEW
To deepen the understanding of CVP performance evaluation during the collaboration of subjects, this research strives to give empirical insights on applied performance indicators. Since only very limited research on this topic exists, the authors chose a qualitative approach to explore the research topic. They conducted multiple interviews in different organizations as a starting point to gain insights into CVP performance evaluation. Already the first interviews showed that organizations are not aware of the indicators they apply. Thus, the main purpose of the interviews is to reveal indicators used by organizations in the automotive industry to evaluate CVPs.
18 in-depth interviews were conducted in 4 automotive or automotive supplier organizations over a 16-month period from April 2018 until July 2019. All companies are located in Germany. The investigated corporates are well-established organizations which regularly collaborate with start-ups in order to access innovation and gain technological insights. Together engaging in about 75 CVP a year, the organizations are chosen due to the proven CV maturity.
The interviews were conducted with 7 experts and focus on the CV process, goals and performance indicators. As interview strategy both, the strategic and operational level of the CVP were investigated. Therefore, the interviewees were chosen according to different levels of closeness to the CVP. This cross-level inter-viewee selection sharpened the understanding. Furthermore, this approach reduced the ex-post rationalization bias. All interviews are based on a semi-structured interview guideline with predefined interview questions. This safeguards a common approach and the discussion of the same topics in the different interviews, thus paving the way for comparison between the interviews and organizations.
The interview had two main goals. Firstly, they served to investigate whether CVP performance indicators are established in organizations. Secondly, they served to reveal implicitly used CVP performance indicators, even if there was no official CV performance measurement system implemented. All interviews lasted between 45 and 120 minutes. In accordance with the interviewee, the interviews were transcribed and then analysed with MAXQDA software. In some cases, the researchers were only allowed to take notes. Afterwards, the collected notes were confirmed with the interviewee.
Each interview started gathering background information about the interviewee and her/his responsibility. After gathering information about the organization's CV history, dependent on the interviewee either strategic information or information about applied CV performance indicators were discussed.
The authors analysed the data collected from the interviews by using a coding system. In a first step the authors deduced relevant codes, based on a literature review and assigned these to the transcribed interviews. Afterwards, further codes were added based on the interview analysis. Codes with similar content got merged into a bigger code and in further iterations into categories. These categories are codes, which built overall indicator perspectives clustering implicitly applied indicators according to their evaluation focus.
Since there is no awareness and no transparency of the indicators for CVP performance evaluation within the interviewed organizations, the semi-structured interview represents the only way, to reveal the indicators in order to reach the article's goal.

RESULTS
Following the described approach, the interviews revealed not only applied CVP performance indicators, but also lead to some basic and foundational results and insights. Before presenting the revealed CVP performance indicators, these basic insights are shared.
As a first result, none of the interviewed organizations applied an objective, clearly defined indicator system for the CVP evaluation. Instead of using an indicator system, the evaluation of CVP is solely based on the gut feeling created out of CVP manager's experience and subjective judgement. However, all interviewed organizations announced that they want and need an indicator system for the evaluation of CVP during the collaboration phase. As revealed by the interviews the major challenge for the organizations is to define what to measure and how to measure.
Nevertheless, based on the in-depth investigation the authors were able to reveal CVP performance indicators which have been implicitly applied. Even though none of these indicators is documented or officially announced, the CVP managers implicitly used these indicators to generate their gut feeling. Consequently, this research contributes to a deeper understanding of CVP evaluation by revealing these implicitly applied indicators. The research findings and thus the revelation of the implicit performance indicators applied by the CVP managers are presented in Tab. 2.
The indicators listed in Tab. 2 are implicitly and unknowingly used by CVP managers to evaluate whether a CVP is beneficial for an organization or not. In total 16 performance indicators were revealed. Yet, CVP managers only based their judgement about a CVP on seven to 14 out of the total indicator list. As an insight from the interview it can be concluded, that CVP managers apply different indicators to draw conclusions on CVP performance. Thus, there is no consistent standard for CVP evaluation during the collaboration of subjects within an organization. Simultaneously, this means that a CVP manager might draw his conclusion about CVP performance on different indicators each time he evaluates the same CVP project, but also when evaluating another CVP project. Consequently, it is likely that a CVP manager's performance evaluation is influenced and biased by his environment.
Looking at the indicators themselves, they can be clustered into 4 categories. These categories are the same for all interviewed organizations. The category "financial indicators" includes two measures. The first indicator was used by four out of four interviewed organizations and describes the ratio between invested capital and planned investment. It measures the investment status and offers insight into the start-up's capital consumption. For this indicator milestones with dedicated budgets are defined. In case the capital consumption exceeds the budget, it is assumed that the startup is inefficient and might not reach the target objective.
The second indicator of the financial category is the corporate's share of venture's revenue. The interviewed organization use this indicator to determine the drive and commitment towards the CVP and the contracting corporate. When the corporate's share of the venture's revenue decreases, it is assumed that the start-ups loyalty decreases as well. This might lead to negatively affect indicators from another category. In conclusion for the financial category, CVP performance evaluation is done by implicitly considering progress indicator, like capital consumption, rather than output indicators, like profit.
The second category encompasses indicators regarding collaboration. In total five indicators regarding collaboration were identified: team stability respectively team fluctuation, management support, response time, number of conflicts and number of social interactions. Considering team stability, the CVP managers evaluate how often changes regarding the team constellation happen. Changes regarding the team are used as an indicator for two aspects: (1) reduction of team performance because of the norming process and (2) withdrawal of resources and/or knowledge. Both affect the CVP performance negatively. All interviewed companies rely on the team stability indicator. Remarkably, team fluctuation is seen negatively by all CVP managers. None of them considered that a change in the team constellation might bring in additional knowledge or specific experience.
A CVP performance indicator which is used by three out of four interviewed organizations is management support. Interpreted slightly different by each organization, this indicator reaches from management's benevolence to management's commitment and prioritization. Mostly the commitment is determined by management's attendance at meetings or postponement of meetings and decisions. The assumption behind is, that with less attendance a CVP becomes deprioritized and loses grip, which will lead to the failure of the CVP. As a result, it can be deduced that in CVP performance evaluation not only the outcome of the CVP team is crucial, but also management involvement. The consideration of management involvement represents a difference to traditional PMS.
Also used by three out of four companies, the response time is seen as an CV performance indicator. Response time means the time a start-up takes to respond to a request. Yet, the start-up is not required to deliver the solution right away. A notification that the request was received, and that the start-up works on a solution is enough. A short response time conveys commitment to the CVP, reliability and thus helps building trust. Trust reduces monitoring activities, leaves freedom for creativity as well as for trial and error. As a result, response time represents an approximator for trust. Even though it does not measure any output or outcome, organizations rely on this indicator. With "number of conflicts" another CVP performance indicator for the collaboration category was revealed. Number of conflicts has two layers, which are evaluated by CVP managers. The first layer represents social conflicts between the start-up's and corporate's teams. The second layer refers to number of problems, which cannot be solved between the teams and therefore are escalated to a higher decision level. A high number of conflicts slows down the CVP collaboration and affects performance negatively. Using this indicator complements the evaluation of CVP performance by adding an approximator to evaluate the CVP atmosphere and thus the chemistry between organizations.
Moreover, one out of the four interviewed organizations applied an indicator measuring the number of social interactions. Social interaction simply refers to the number of "off the job" socializing. It is seen as a performance indicator directly affecting the CVP collaboration. If the CVP team has a strong personal bond, they share information and support each other as quickly as possible. This personal bond beyond the job is tried to grab by measuring the number of social interaction and events after working hours. In summary of the collaboration category, the focus of the applied indicators again lies on progress indicators. Mostly, approximators are used which try to make trust quantifiable. Interestingly, not only the pure CVP output performance is considered, but also the involvement of management as a key element for CVP success.
The next category refers to innovation. As described above, CVP focus on innovation and the window on technology in the case of the investigated organizations. Still, only three performance indicators were revealed: number of impulses, number of features tested/ updates and off the job time. Three out of four interviewed companies use "number of impulses" as indicator for CVP performance. With this indicator the organizations try to monitor the CVP innovativeness. It is assumed that a high number of impulses increases the likelihood of innovation and success. Referring to this interpretation an idea represents an impulse as much as a proposal of a joint workshop or an innovative solution. The only restriction is, that impulses must refer to the product or service the CVP team is working on. In contrast, it embodies a loss of commitment and effort, if the number of impulses decreases. For CVP managers this indicates that the CVP might not be able to achieve the CVP innovation goal. The number of features tested/qualified updates represents another CVP performance indicator. Three out of four interviewed or-ganizations use this indicator to evaluate the quality of the start-up's deliverables. The corporates compare successfully tested features with the total features tested. The closer the ratio is to 1, the better the quality. Apart from that, the number of qualified updates is considered. Having current project management data and proper documentation available (called qualified updates) is as important as the features tested. In case the quality decreases, it is assumed that either the start-up is not capable of delivering the innovation or that the commitment decreasing. Both aspects reduce CVP performance. Interestingly, all interviewed companies which use this indicator also used the indicator "number of impulses". Another innovation indicator, which is applied by one corporate, measures off-the-job time. Off-the-job time sums up the time, the CVP team is not working on the innovation, but is caught in meetings or in administrative tasks.
The probability of innovation and a successful CVP decreases with a high off-the-job time.
In conclusion, it can be deduced that the innovation category tends to rather focus on output than on progress. This is a difference towards the other CVP performance evaluation categories. Nevertheless, no traditional lagging indicators like number of patents are used. The applied indicators rather try to anticipate the likelihood of innovation by referring to impulses and elements of the desired innovation. Simultaneously, it can be deduced that the assumption behind is, that innovation is plannable and therefore dedicated features can be determined ex-ante.
The last category uses a process perspective. It encompasses six performance indicators to evaluate CVPs at an early point of time. Firstly, the number of on-time deliveries is measured. Therefore, milestones and assigned deliverables are predefined. If the results are delivered at the pre-defined milestone, the CVP is evaluated as successful. The indicator simply builds the ratio between promised deliverables and actual deliverables based on the milestone plan. The closer the ratio is approaching 1, the better the performance. Four out of four interviewed organizations deployed this performance indicator. Asking the interview question, which revealed indicator is the most important one, all interviewees answered that on-time delivery is most crucial. This indicator implies that a CVP is plannable and has a clearly defined outcome. also implies, that Even though ontime delivery implies plannability and a clearly defined structure of innovation Another process indicator is the number of implemented best practices. Three out of four investigated companies apply this indicator to implement improvements. It is assumed that the relationship and way of working improves by a high number of best practices. Simultaneously, implemented best practices increases the likelihood of innovation and therefore CVP performance. As a result, this indicator aims to improve one of the main issues in CV, namely the disconnection between corporate and startup regarding processes.
Time-to-market is essential for innovations. Consequently, three out of four interviewed organizations use the number of slow decisions or not available incidents of the start-up's management as a CV performance indicator. Both, slow decisions and unavailability of the start-up's management implicitly mirror, that the priority of the CVP is not very high. In case the CVP has highest priority, the startup's management will find the time to cope with decision or requests from the corporate. This indicator is strongly interlinked to the collaboration category and the indicator management support.
A quite popular indicator represents the number of information exchange sessions. Three interviewed organizations use this measure to evaluate the CVP. The assumption is, that meetings with the purpose to exchange information help to provide relevant information and thus augment the likelihood of innovation. Moreover, information exchange sessions are associated with joint problem-solving. Thus, the interviewed organizations argue that frequent meetings increase performance. As a result, the basic requirement of collaborations is targeted with this indicator. Only if the information flow between entities is enabled, innovation leading to CVP success is possible.
Two out of three interviewed organizations complement this indicator with the exchange session attendance rate. Since measuring only the number of information exchange sessions is not representing the entire picture, these two organizations also measure whether the relevant experts participate (assuming that the meeting participants are relevant for the session). A high attendance rate safeguards information exchange between relevant experts and thus fosters innovation. Consequently, good information exchange is likely to increase the CV performance.
The last process indicator which was revealed is only used by one interviewed organization. It is the number of celebrated successes. This indicator refers to two topics. On the one hand, the number of celebrated successes only increases if the innovative outcome for the milestones has been achieved. Thus, representing the overarching success of the CVP. On the other hand, this indicator implicitly supports building the relationship between the teams, since a joint celebration fosters the personal bond and increases motivation. Implicitly the number of celebrated successes sums up the overall performance and thus represents a CVP performance indicator.
Summarizing the process category, this category encompasses the most indicators in comparison to the other three categories. The implicitly applied indicators represent a mixture of traditional indicators like on-time delivery and more dynamic indicators like number of not available incidents. The indicators are rather used to monitor the development instead of leading to a digital decision stating whether the goal is achieved or missed.
In conclusion, all interviewed organizations implicitly apply indicators to evaluate CVP during the collaboration of subjects. None of the organizations uses a defined, standardized set of indicators, nor are thresholds for the implicitly applied indicators defined. Only due to the interview the authors were able to reveal the applied CVP performance indicators. The revealed indicators can be categorized in 4 categories which are the same for all interviewed organizations: finance, collaboration, innovation and process. Apart from that, the interviewed organizations use quite similar indicators with only little variance. All investigated companies implicitly evaluate the start-ups and CVP based on 16 financial and strategic aspects. The finance category is the least important category for all interviewed organizations. Even though, innovation represents the main goal of the CV collaboration, only three indicators directly refer to this category. Interestingly, most applied indicators embody progress indicators, which rather approximate the outcome. Most of the indicators try to capture qualitative performance elements and transfer them in an approximated quantitative measure. In comparison to traditional performance measurement, the CVP indicators not only focus on the team's performance, but also considers management performance by measuring for example availability. As an insight, all implicitly applied CVP performance indicators contribute to build trust between the corporate and the start-up. These results help to understand the mechanisms how corporates evaluate CVP performance during the collaboration of subjects. Moreover, the revealed indicators contribute to a deepened understanding of CVP evaluation during the collaboration phase.

DISCUSSION
After having presented the empirical results, a discussion of the results, especially the 16 performance indicators applied for CVP evaluation during the collaboration phase gives the opportunity for further insights regarding the research topic.
In the present study the author coped with the lack of missing empirical research and empirically examined the applied indicators to evaluate CVP. It became obvious that none of the interviewed companies established a formal CVP evaluation framework for the collaboration phase with defined performance indicators. Still, all interviewed organizations long for such a framework and desire a guideline with defined indicators. As the empirical research results show, the investigated organizations implicitly apply 16 different CVP performance indicators during the collaboration phase. These indicators are discussed in the light of existing indicators from various research areas.
Existing literature focuses on traditional performance measurement methodologies and indicators. Yet, the usage of established measures -not developed for CV -has its limitations (Banik, 2011;Keil, 2000). Existing frameworks focus on measuring performance of single businesses (Pekkola and Ukko, 2016;Westphal et al., 2010). Frameworks which focus on CVP evaluation during the collaboration phase and thus cross-company management are hardly existing (Westphal et al., 2010).
Available CV performance measurement research almost exclusively evaluates CV success determinants or CV strategies. Systematically structured operational indicators, which evaluate CVP during the collaboration itself are lacking (Dauderstädt, 2013;Faisst, 2005;Kollmann and Kuckertz, 2010). Literature acknowledges, that a CV performance evaluation framework is crucial to increase transparency and effectiveness regarding CVP (Faisst, 2005). Therefore, CV specific requirements need to be met (Westphal et al., 2010;Faisst, 2005). To develop a CV specific and practically usable performance indicator set, empirical insights help to understand CVP performance evaluation.
Even though, implicit performance indicators have been identified, three of the four indicator categories are quite vague -with exception of the financial category. In general, a huge point for critique is that all interviewed organizations only refer to managers' gut feelings when evaluating CVP performance. There are no clearly defined indicators, thresholds or guidelines for orientation in order to define good CVP performance. This holds true for all indicators. This overarching result will not be mentioned further in the discussion. Only selected indicators with need for discussion are presented in the following.
Looking at the collaboration indicator category, team stability for example is used by four out of four interviewed organizations. Changes in the CV team are assumed to have negative impact. Having a solid team aligns with existing venture capital literature. However, venture capital looks at the team constellation ex ante in order to determine whether a team is capable of being successful. In this context literature states that a stable team positively impacts performance (De Clercq et al., 2006). Yet, during a CVP this holds only partially true. Sometimes, changes in the team bring new expert knowledge into the CVP. Thus, in case of high-quality staffing the change of team constellation might even increase CVP performance. Since experts oftentimes add value only to specific topics or phases, a change of the team constellation might even be necessary. Consequently, counting the number of team changes is not enough. In order to be reliable, only random team changes should be considered negatively. From the authors point of view, this indicator should not be adopted from venture capital literature and not be considered in CVP performance evaluation during the collaboration phase.
Top management support is stated to be one of the most important indicators for CVP performance, not only by the interviewed organizations, but also in literature (Schween, 1996;Banik, 2011). Still, the indicator top management support should be discussed critically. Top management support is very important for successful CVP. However, it must not be confused with approval of all CVP requests. Moreover, there is no clearly defined way of measuring top management support. From the authors point of view, top management support could rather be measured by the indicator "not available incidents". Since top management support is a vague term, the author recommends to clearly define measurable events in order to consider this indicator for CVP performance evaluation.
Two out of four interviewed organizations use the indicator "number of social interactions". This indicator is also discussed controversially in literature (Maula, 2001). From the authors point of view, this indicator could lead to wrong evaluations. Implicitly, the number of social interactions might increase performance. Yet, setting this indicator in stone would also lead to a more negative evaluation of CVP, which simply does not offer the possibility to visit joint events -for example because of geographical distance. Moreover, social interaction could also lead to negative effects, when people recognize at social events that they do not trust one another. Based on the geographical influence and the missing focus of a clear goal when considering this indicator, the authors recommend further investigation whether this indicator is suitable for CVP evaluation.
Even though all CVP focus on innovation, basic definition and thresholds are missing to have conclusive innovation indicators. Based on the vagueness of the implicitly applied indicators, it is almost impossible to deduce reliable conclusions about CVP performance. From the authors perspective at least a clear process of how to define thresholds for each CVP and a up front acceptance of the thresholds between the CV parties needs to be given. Defining a standard threshold without considering specific CVP does not make sense, since each CVP has different requirements.
Looking at the process category, the before mentioned challenge of missing thresholds and clearly defined indicator hamper objective measurement. Especially when focussing on the number of slow decision/not available incidents two topics need to be critically discussed. Firstly, it is not defined what a slow decision is and how it is measured. On the one hand, "slow decision" is decision specific and depends on the individual perception of everyone. On the other hand, even if a slow decision could be defined, there would still be the question how to measure it referring to starting point. This discussion also emerges in entrepreneurship literature (Sadowski, 2001;Toschi, 2009). In entrepreneurship literature this indicator is seen as the basis for fast innovation -the key goal of CV. From the authors perspective, measuring not available incidents would be easier to measure in comparison to slow decisions. Still, it needs to be defined when non-availability can be counted as not available incident. Is it a not available incident when the decider is on vacation? Does a not available incident refer to all events or only for important decision with time pressure? How much time needs to pass until missing availability can be counted as not available incident? So far, this indicator plays only a minor role, however it offers quite some insights into CVP performance. Consequently, the authors recommend analysing the practical applicability of the indicator number of not available incidents.
Another process indicator represents the number of exchange sessions. Not only for the interviewed companies, but also for various literature streams this indicator is relevant for performance evaluation (De Clercq et al., 2006;Dauderstädt, 2013;Camarinha-Matos et al., 2009). Yet, from the authors point of view, solely looking at the number of exchange sessions might distort the picture. Firstly, the amount of necessary exchange sessions depends on the project. Too many exchange sessions for example even might decrease performance, since the CV team spends much time in nonvalue-adding meetings. Thus, a suitable number of sessions needs to be defined. Secondly, exchange sessions themselves are not enough to transfer relevant information. Also, the relevant experts need to be present. Consequently, the number of exchange session is only relevant in combination with the attendance rate of relevant experts and managers (Camarinha-Matos et al., 2009;Westphal et al., 2010). Only with the right people, quality of the information exchange sessions can be guaranteed. From the authors perspective, these indicators only give insights into CVP performance when considered together.
Even though the authors revealed the indicator "number of celebrated successes", this indicator seems to be a practical appearance, not discussed in in literature so far. At first sight, the indicator seems to correlate with the number of social interactions. However, the number of celebrated success has a broader perspective and from the authors perspective is more relevant for CVP performance evaluation. Celebrated successes include various components. Firstly, it means that endeavours have been successfully finished. Secondly, it means that the milestone plan including the deliverables have been successfully achieved. Thirdly, the number of features tested/updates was great enough to achieve the targeted outcome, which also implicates that the number of impulses and off-the-job-time was in balance. Fourthly, it strengthens the personal bonding between the organizations, which would also partly cover the indicator number of social interactions. Fifthly, it increases motivation of the CV team which inherently influences performance positively. To the knowledge of the author, this indicator is its infancy and further investigation is needed.
All in all, this empirical research reveals practically applied indicators. In the discussion part of the paper, the authors shed light on the weaknesses and insufficiencies of the implicitly applied CVP performance indicators for the collaboration phase. By revealing the indicators and discussing their weaknesses, this research contributes to CV literature and a deeper understanding of CVP performance evaluation.

CONCLUSION
Even though the importance and value-add of CVP has been widely acknowledged in literature, the understanding of CVP performance evaluation during the collaboration phase is in its beginning (Dauderstädt, 2013;Husted and Vintergaard, 2004;De Clercq et al., 2006). Empirical studies coping with CVP performance evaluation during the collaboration phase are lacking. Therefore, this study addressed the need for empirical data on practically applied CVP performance indicators. To reveal these indicators, the authors conducted 18 in-depth interviews at four automotive companies.
The results show that even though the interviewed organizations longed for an objective and clearly defined set of indicators, the actual evaluation was based on subjective gut feeling. The authors made visible, which factors lead to the CV managers' gut feelings and summarized implicit indicators. In total 16 CVP performance indicators, used for the evaluation of CVP during the collaboration phase have been revealed. They all align with one of the four categories: finance, collaboration, innovation and process.
The study contributes to existing literature by expanding theoretical knowledge of CVP performance measurement during the collaboration phase. As one of the first in-depth studies of the CVP collaboration phase, it revealed implic-itly applied performance indicators. Also, this empirical research explains why the revealed indicators are applied. Finally, this research contributes to literature not only by revealing applied indicators, but also by comparing these indicators with indicators extracted from various literature streams and shedding light on the weaknesses and insufficiencies in the context of CVP.
From a managerial perspective this research contributes by proposing four CVP performance perspectives and the revelation of 16 CVP performance indicators applied by various organizations in order to set the basis for defining a CVP performance measurement framework. By explaining applied indicators and critically examining them, the authors give an orientation for managing CVPs during the collaboration of subjects.
Nevertheless, interviews can only be a starting point of the research. One of the research's limitation is the restricted generalizability. In order to establish a proven set of CVP performance indicators for the collaboration phase, quantitative research is needed to validate and generalize the research's results. Moreover, further research is needed to investigate the transferability of the revealed CVP performance indicators to different industries.