Validación Inter-operador de Videobserver™

Inter-operator reliability of Videobserver™

Validação Inter-operador de Videobserver™

 

 

Fortes, A.M.^1,2, Gómez. M.A.², Hongyou L.³, y Sampedro, J.²

¹Faculty of Physical Activity and Sport Sciences, Polytechnic University of Madrid.
²Sports School of Rio Maior, Polytechnic University of Santarém.
³College of Physical Education and Training, Shanghai University of Sport

Correspondence

 

 

---

ABSTRACT

The purpose of this study was to evaluate the degree of inter-observer reliability for the sport analysis company VIDEOBSERVER™. The football match corresponding to the 2014 World Cup final was used as the sample for the study. Data were collected on 28 players, including the actions of the goalkeepers and the players substituted during the match. This study was carried out in four phases: (i) a revision of the literature was performed to define the theoretical frame-work and the procedures to be followed; (ii) an observation system was created with the help of two experienced sports analysis technicians; (iii) a pilot study was carried out with the aim of verifying the operativity of the observation system; and (iv) the main study of inter-observer reliability was developed. The values of the kappa index, the standardized typical error and the intra-class correlation coefficient were used for the statistical analysis. The results showed a high level of inter-operator reliability, a statement which is supported by the high values of the intra-class correlation coefficient (both at the individual and collective level) and the small standardized typical error which were found in this study. This study adds an innovative element of analysis consisting in the fact that the evaluation was carried out not only at the collective level but also at the individual level.

Key words: Videobserver, match analysis, performance indicators, reliability, soccer.

---

RESUMEN

La realización de este estudio se llevó a cabo con el propósito de evaluar el grado de fiabilidad inter-operador de la empresa de análisis deportivo VIDEOBSERVER™. Como muestra para la investigación se utilizó un partido de fútbol de la final de la Copa del Mundo del 2014. En la toma de datos se recogieron los datos de 28 jugadores, incluyendo las acciones de los porteros y de los jugadores sustituidos durante el partido. El desarrollo de esta investigación se llevó a cabo en cuatro fases: (i) Se realizó una revisión bibliográfica para definir el marco teórico y los procedimientos a seguir; (ii) De este estudio se creó un sistema de observación con la ayuda de dos técnicos deportivos superiores; (iii) Se realizó un estudio piloto con el objetivo de verificar la operatividad del sistema de observación; (iv) Se ejecutó el estudio principal de validez observacional inter-observadores. Para realizar el análisis estadístico se utilizaron los valores de índice kappa, error típico estandarizado y coeficiente de correlación intra-clase. Los resultados obtenidos muestran una elevada fiabilidad a nivel inter-operador, esta afirmación se encuentra sustentada en los elevados valores del coeficiente de correlación intra-clase (tanto a nivel individual como colectivo) y el reducido error típico estandarizado que se obtuvieron en este estudio. Este estudio añade un elemento de análisis novedoso, que consiste en el hecho de que en este estudio realizado la validación no solo a nivel colectivo sino que también se realiza a nivel individual.

Palabras clave: Videobserver, Metodología observacional, indicadores de performance, Validacion, Futbol.

---

RESUMO

Este estudo foi realizado com o objetivo de avaliar o grau de fiabilidade inter-observador da empresa de analise desportivo VIDEOBSERVER™. A amostra utilizada nesta investigação consistiu num jogo de futebol da final da Copa do mundo de 2014. Na recolha de dados registaram-se as informações de 28 jogadores, incluindo as ações dos guarda redes e jogadores substituídos durante o jogo. A construção desta pesquisa foi realizada em quatro fases: (i) Foi realizada uma revisão da literatura para definir o marco teórico e os procedimentos a serem seguidos; (ii) Criação de um sistema de analise com a ajuda de dois técnicos de superiores de desporto; (iii) Realizou-se um estudo piloto com o objetivo de verificar o funcionamento do sistema de observação; (iv) Execução do estudo principal estudo de validade observacional inter-observador. Para realizar a análise estatística utilizou-se os valores de índice kappa, o erro típico e coeficiente de correlação intra-clase. Os resultados obtidos mostram um elevado nível de fiabilidade inter-operador, esta afirmação é suportada pelos valores elevados do coeficiente de correlação intra-clase (tanto a nível individual como coletivo) e o reduzido erro-padrão obtido neste estudo. Este estudo adiciona um elemento de análise inovadora, que consiste no fato de que, no presente estudo não só de realizou uma validação a nível coletivo, mas também é feito a nível individual.

Palavras chave: Videobserver, Metodología observacional, Indicadores de performance, Validação, Futebol.

---

 

Introduction

Game analysis is commonly used in many sports, as it is an essential process which permits coaches to collect objective information that they can use as a performance aid (Hughes & Franks, 2004; Hughes & Bartlett, 2002). It is fundamental to understand the internal logic of the sport to be analysed (Sampedro, 1999).

Such analysis permits the objective collection and examination of the behavioural events which occur during the competition in a complete and systematic way (Carling et al., 2005).

The main objective in match analysis is to identify the sources of the strengths of the teams, which can be developed later, and their weaknesses which indicate the specific areas for improvement. In the same way, a coach analysing an opponent team can use the data to identify the different ways to counteract the strengths of this team and to exploit its defects and limitations (Carling, Reilly & Williams, 2008).

There exists a wide variety of techniques for carrying out game analysis which can be performed with or without the aid of a computer. Non-computerized techniques consist in real time observation and notation; these methodologies are very laborious (due to the time it takes to analyse and collect the data) and depend on factors like the accumulated experience of the observer and their perspective with regard to the match. Computerized techniques can be classified, according to Ballesta, García, Fernández and Alvero (2015), by the type of monitoring that they perform on the player, and can be divided into: via GPS, with video recording and editing, and with automatic assisted image recording. (Ballesta et al., 2015; Barris & Button, 2008; De la Vega-Marcos et al., 2008).

The GPS system is used to quantify the training load and to record different types of movements in real time in the practice of football, being a reliable system for recording the movements of the player, but it has the disadvantage that it is not allowed by the FIFA regulations and thus its use is limited to the training sphere as it cannot be incorporated into the gear during official competitions. The systems for the automatic monitoring of the image and assisted movement analysis make it possible to obtain information in real time and a large volume of data but imply a high cost in material and time due to the manual adjustments which have to be made.

With regard to the applications for video recording and editing, they facilitate the recording of the action, but the quality of the results obtained may be affected by the subjectivity and degree of training of the observer (Ballesta et al., 2015; Barris & Button, 2008).

Hughes and Franks (2004) suggest that an inter-observer reliability analysis should be carried out on all computerized analysis systems (to guarantee the reliability of the data), and it was following this suggestion that the present research was undertaken, as this study aimed to evaluate the degree of correlation between variable we want to analise and test the reliability (Sampedro, 2013 ) for the sport analisys company Videobserver.

 

Material and Method

Software

The instrument used for the validation in the present study was the Videobserver™ software (Version 1.3.68, Lisbon, Portugal).

This software collects the data on the match through a graphics interface which contains two windows, one window shows the video of the match and the other shows a graphics representation of the playing field. To introduce data into the program it is necessary to click the left mouse button twice. Once to locate the area on the playing field where a determined action is taking place and the player that is executing it, and another to select the type of action that the player carried out.

 

 

There are actions where the software offers the option of qualifying the action as: "Successful", "unsuccessful", "Intercepted", etc. and in these cases the observer has to click on the qualification they feel most suitable. At the bottom of the interface there is a bar which shows when a determined action was executed and what type of action was recorded.

This software permits the data collected to be verified at any moment by clicking on the bottom bar or the field actions window. It allows adding, omitting or editing the actions, which should be analysed using a button, labelled "personalized actions", this function permits more flexibility in the analysis altering at any time the analysis system to be used.

 

Sample and Procedures

Characterization of the sample

The match chosen to carry out this study is a recent match corresponding to the final of the World Cup played between Germany and Argentina on 13th July 2014. Data were collected on 28 players and included the actions of the goalkeepers and the players who were substituted during the match.

The following criteria were used to define the study sample:

a) The match to be analysed had to be from a competition among national teams, this criterion was used because in this type of competition there is a high level of competitiveness as to classify for the final in this championship the teams need to pass a preliminary phase which only permits the best teams to continue (Silva, 2007). This is why we used a World Cup match played in 2014. To secure a great level of the data were chosen two elite teams. The criterion used to choose the teams was created by Silva (2007).

b) The field players chosen for the individual analysis of their actions were selected at random.

Procedures

This study was carried out in four phases, in the first phase of the study a bibliographic review was conducted which permitted the definition of the theoretical frame-work and the procedures to be carried out as well as the determination of the study design.

In the second phase the analysis system was created using the following procedure: first the researcher conducted a bibliographic review of the authors: Braz and Borin (2009), Castelo (2009), Hughes and Bartlett (2002), Garganta (2009), Silva (2007), Ribeiro (2008), Barros et al (2002) and Reina and Hernández (2012) which permitted identifying the variables needed to create the system. After creating the analysis system two qualified experts in football (high level sports analysis technicians) were recruited to perform a triangulation of researchers to permit the observation system to be improved.

Once the analysis system had been created, it was possible to proceed to the third phase of the research, in which a pilot study was carried out in the company headquarters located in Lisbon. The pilot study was performed with the aim of verifying the operativity of the observation system.

The pilot study followed the following protocol: first the company configured the software so that the analysis could be performed, then the operators were located in the same area to guarantee that they would be exposed to the same conditions and they were informed that they had to carry out a simultaneous analysis of both teams participating in a qualifying match for the European football championships in 2016 based on the analysis system created by the researcher. The researcher was in the same room but he just timed the operators as they carried out the analysis.

Once the pilot study had finished the fourth and last phase was executed in which the main study was carried out using the same operators, the same location and the same protocol as the pilot study.

The two operators who participated in this study were expert operators (Liu et al., 2013) as they had experience of 1.5 and 2 years analysing football using software, and took part both in the pilot study and the main study.

The approval of the experimental test was obtained from the Company of Videobserver™ (Lisbon, Portugal) before the experiment.

 

 

Variables

O'Donoghue (2006) states that any play action or variable should be accurately defined so that they are not ambiguous, and bearing this criterion in mind it was decided to create and define the analysis system which gave rise to the following set of variables:

Tactical schemes

• Corner kick - This is awarded when the ball leaves the field of play by crossing the goal line (either on the ground or in the air) without a goal having been scored, having been last touched by a defending player (FIFA., 2015).

- Goal- when the ball crosses the line between the goalposts (Sáez, 2007).

- Save- The action when the goal keeper stops the ball shot towards the goal (Sáez, 2007).

- Out of play- a ball which goes out of the play area (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

• Direct free kick - shooting at the rival's goal as punishment for a foul in which the player shoots with no intermediary (Sáez, 2007);

- Goal- when the ball crosses the line between the goalposts (Sáez, 2007).

- Save- The action when the goal keeper stops the ball shot towards the goal (Sáez, 2007).

- Out of play- a ball which goes out of the play area (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

• Indirect free kick - shooting at the rival's goal as punishment for a foul in which the player cannot shoot at goal directly, but has to pass the ball to a teammate (Sáez, 2007);

- Successful- realization of the indirect free kick (Sáez, 2007).

- Unsuccessful- frustrated or not achieved (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

• Throw-in - a player putting the ball in motion with two hands (from behind the head) and from outside the side line towards a teammate (Sáez, 2007);

- Successful - realization of the throw-in (Sáez, 2007).

- Unsuccessful- frustrated or not achieved (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

• Goal kick - This is when the ball leaves the field of play by crossing the goal line (either on the ground or in the air) without a goal having been scored, having been last touched by an attacking player (FIFA., 2015).

- Successful - realization of the goal kick (Sáez, 2007).

- Unsuccessful- frustrated or not achieved (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

Variables related to attacking

• Assist- This is the last pass to the receiver who scores a goal (Liu et al., 2013).

• Cross- Any ball kicked or headed into the offensive zone from far away (Liu et al., 2013).

• Shooting- Technical tactical action by the possessor of the ball with the objective of sending the ball into the rival's goal (Castelo, 2009).

- Goal- sending the ball across the goal line (Sáez, 2007).

- Save- The action when the goal keeper stops the ball shot towards the goal (Sáez, 2007).

- Ball sent out of play - the ball leaves the playing area (Sáez, 2007).

- Intercepted- preventing the ball from following its trajectory (Sáez, 2007).

• Off side - A player is off side when attacking when they are further forward than all the other opposing players except one (the opposing goal keeper) (Liu et al., 2013).

• Loss of the ball - When the attacking team loses possession of the ball through the action of the defending team (Liu et al., 2013).

• Penalty- A penalty kick is awarded if a player commits an offence in their own goal area, independently of the position of the ball (FIFA., 2015);

- Goal- sending the ball across the goal line (Sáez, 2007).

- Save- The action when the goal keeper stops the ball shot towards the goal (Sáez, 2007).

- Ball sent out of play - the ball leaves the playing area (Sáez, 2007).

Variables related to defence

• Foul- Any action penalized as a foul by the referee (Liu et al., 2013).

- Fouls received.

- Fouls committed.

• Penalization- Disciplinary measures against players who commit offenses worthy of being cautioned (Liu et al., 2013);

- Yellow card - means a caution (Sáez, 2007).

- Red card - means being sent off (Sáez, 2007).

• Recovery of the ball - When by its actions the defending team recovers possession of the ball (Liu et al., 2013);

• Own goal- A goal scored by the defending player in their own goal (Sáez, 2007);

• Tackle- Action carried out by a defender who steals the ball from the attacker (Liu et al., 2013);

• Interception- The blocking of a shot by a defender. This action should be assigned to the player who intercepts the shot (Liu et al., 2013).

 

Statistical Analysis

 

 

The index used to measure inter-observer reliability was the kappa index. The kappa index has been used in previous studies and has been proved to be the most reliable for calculating the agreement among observers (O'Donoghue, 2015). The method suggested by O'Donoghue (2010) and Robinson and O'Donoghue (2007) was used to calculate the kappa value and is calculated with the following equation:

To interpret the calculated Kappa values it was necessary to have recourse to the interpretation agreement suggested by Altman (1991). This agreement states that: when the value of Kappa is < 0 the possibility of agreement is inverse; from 0.01 to 0.20 concordance is poor; from 2.1 to 0.40 the degree of agreement is limited; from 0.41 to 0.60 the agreement is moderate; from 0.61 to 0.80 there is good agreement among the operators; and from 0.81 to 0.99 the agreement is very high.

A spread sheet of the reliability of observation of recordings developed by Hopkins (2000) was used to calculate the absolute values (mean, changes in the mean, standardized typical error and intra-class correlation coefficient) of the actions recorded in the match, where to interpret the values of the typical standard error the following values are considered: < 0.20 trivial; 0.21-0.60 small; 0.61-1.20 moderate; 1.21-2.00 large; 2.01-4.00 very large; > 4.00 extremely large (Hopkins, 2000; Smith & Hopkins, 2011; O'Donoghue, 2007). The interpretation of the intra-class correlation values was as suggested by Prieto, et al., (1998) which consider the following ranges: <0.30 poor or null; 0.31-0.50 low; 0.51-0.70 moderate; 0.71-0.90 good; >0.90 very good.

As the teams were codified by two different operators, the absolute values were divided into two groups, one for the German team and the other for the Argentinean team. The value of the standardized typical error was analysed according to the following ranges: < 0.20 trivial; 0.21-0.60 small; 0.61-1.20 moderate; 1.21-2.00 large; 2.01-4.00 very large; > 4.00 extremely large (Hopkins, 2000; Smith & Hopkins, 2011; O'Donoghue, 2007).

 

Results

The results shown in Table 3 reveal that there were 679 agreements in the actions codified by the two independent operators, 357 agreements in the actions of the German team and 322 agreements in the actions of the Argentinean team. Tables 5 and 6 present detailed information on the agreements. The mean difference in event recording was 0.06 ± 0.04 s. The Kappa values reached a good inter-operator agreement level of 0.92 for Germany and 0.80 for Argentina.

 

 

In the individual analysis of the selected players there were 189 agreements. Table 4 shows that there was some variety in the Kappavalues at the individual level. Neuer revealed a perfect Kappa value of 1, while S. Romero showed a Kappa value of 0.70. The defences revealed a very high level of agreement with Boateng showing a Kappa value of 0.89 and Garay 0.95. With regard to the mid-fielders Kross had a good level of inter-observer agreement with a Kappa value of 0.73 and with regard to Mascherano the operators attained a very high level of agreement with a value of 0.85. The values for the forwards were very varied as for Müller the value was 0.92 and for Messi 0.70. The agreements of actions at the individual level codified by the operators are shown in detail in Tables 7, 8, 9, 10, 11, 12, 13 and 14.

 

 

 

 

Secondly, we present the results obtained with the Hopkins spreadsheet (2000) to analyse the reliability of independent observers. Table 15 shows the standardized typical error for the actions codified by the operators which varied from 0 to 0.63. The intra-class correlation coefficient varied from 0.76 to 1.00 which shows a high level of reliability.

 

 

In relation to the actions codified at the individual level, Table 16 shows that the values for the standardized typical error varied between 0 and 0.33, as well as the typical standard error, the results show that the intra-class correlation coefficient reached high levels of reliability which varied from 0.91 to 1.00.

 

 

Discussion

The main aim of this study was to verify inter-operator reliability for VIDEOBSERVER™ football analysis company, and the results obtained in the analysis allow us to state that this tool presents a high level of reliability with regard to inter-operator analysis, as can be seen in the high values of intra-class correlation (both at the individual and collective level) and the small standardized typical error obtained, being similar to those referenced in previous reliability studies on observational software for football (Liu et al., 2013).

The intra-class correlation value for Germany in all its actions was 0.99, the standard typical error was 0.034 and the Kappa value was 0.92. The value for Argentina in this study was 0.88 for total actions with a standard typical error of 0.35, and a Kappa value of 0.80. These figures show high positive values of reliability in the analyses of both observers according to the specific references of Altman (1991) and Hopkins (2000).

At the individual level the selected players revealed the following values: the goal keepers showed a difference in the Kappa values of 0.3 (Neuer =1; S. Romero=0.70) and in relation to the reliability of the codified actions a correlation was obtained of 1 with a typical standard error of 0.001. With regard to the defences a difference of 0.06 was shown in the Kappa value (Garay =0.95; Boateng =0.89) with a correlation of 0.99 for Boateng and a typical standard error of 0.09. For Garay a correlation was obtained of 1 with a typical standard error of 0.001. With regard to the mid-fielders there was a difference of 0.12 in their Kappa value (Mascherano=0.85; Kroos =0.73) with a correlation of 1 and a typical standard error of 0.001 for Mascherano and with a correlation of 0.91 and a typical error of 0.33 for Kross. Finally the forwards showed a difference of 0.22 in their Kappa value (Müller =0.92; Messi =0.70) where there was a correlation of 0.99 for the two players and a typical standard error of 0.08 for Müller and de 0.11 for Messi. Similarly to the values for the teams, these were very reliable following the Kappa index values (Altman, 1991) and also considering the Intra-class correlation coefficient according to Hopkins (2000).

A significant degree of variability can be seen in the individual Kappa values which could be due to the fact that, at the individual level, the players have a limited number of actions, which means that any disagreement significantly affects the Kappa value.

As well as analysing the agreements, it is also necessary to analyse the disagreements and to discover why they occur. In this study the disagreements could be due to two aspects: one is the fact that in some occasions the play actions are not transmitted in their entirety due to repetitions of actions or changes of perspective, which makes it difficult to codify some actions and increases the probability of assigning the wrong code, and also the fact that in some shots the camera is far from the action making it difficult to identify the player involved.

Taking all the values obtained, the VIDEOBSERVER™ recording process has been shown to be reliable, a fact that can be attributed to three factors, one is the rigor with which the analysis system was created which permitted the operators to clearly understand what actions had to be codified, and the fact that the participants in this study were experienced operators implying that they had a high level of training in match analysis using this tool (Liu et al., 2013). The third and last factor is the facility and flexibility which this tool offers greatly enhancing the codification of the different play actions.

 

Conclusion

The present study on the validation of VIDEOBSERVER™, showed that the data collected for this company presents a high degree of inter-observer reliability.

Based on these conclusions drawn from the data analysed this play analysis tool can be said to be reliable for use in academic research, published articles and the analysis of training and competitions.

 

Practical Applications

This study had an added innovative element which was the analysis made of play actions on the field performed by individual players, an aspect which was not present in any of the published articles consulted for this research, from our point of view, the introduction of individual analysis in the validation Inter-operator provides this study a useful view, creating possible new lines of research.

 

References

1. Altman, D.G. (1991). Practical Statistics for Medical Research. London: Chapman & Hall.         [ Links ]

2. Ballesta Castells, C., García Romero, J., Fernández García, J., & Alvero Cruz, J. (2015). Revisión/Review Métodos Actuales De Análisis Del Partido De Fútbol Current Methods Of Soccer Match Analysis. Retrieved url.         [ Links ]

3. Barris, S., & Button, C. (2008). A review of vision-based motion analysis in sport. Sports Medicine, 38(12), 1025-1043.         [ Links ]

4. Barros, R. M. L. d., Lima Filho, E. C., Freire, J. B., Bergo, F. G., Cunha, S. A., & Bremzikofer, R. (2002). Sistema para anotação de ações de jogadores de futebol. Revista Brasileira do Ciências do Movimento, 10(2), 7-14.         [ Links ]

5. Braz, T. V., & Borin, J. P. (2009). Análise quantitativa dos jogos de uma equipe profissional da elite do futebol mineiro. Revista da Educação Física/UEM, 20(1), 33-42.         [ Links ]

6. Carling, C., Reilly, T., & Williams, A. M. (2008). Performance assessment for field sports. London: Routledge.         [ Links ]

7. Carling, C., Williams, A. M., & Reilly, T. (2005). Handbook of soccer match analysis: A systematic approach to improving performance. London: Psychology Press.         [ Links ]

8. Castelo, J. F. (2009). Fútbol: estructura y dinámica del juego. Barcelona: Inde.         [ Links ]

9. De la Vega Marcos, R., Díaz, S. D. V., Rico, A. M., & Hernández, A. M. (2008). Una nueva herramienta para la comprensión táctica en el fútbol. International Journal of Medicine and Science of Physical Activity and Sport (30), 130-145.         [ Links ]

10. Desporto, M. E. E. F. E. (2007). Edmundo José De Oliveira Silva. Universidade De Trás-Os-Montes E Alto Douro. (Thesis) poner enlace.         [ Links ]

11. Fédération Internationale de Football Association (2015). Laws of the game. Zürich: FIFA .         [ Links ]

12. Garganta, J. (2009). Trends of tactical performance analysis in team sports: bridging the gap between research, training and competition. Revista Portuguesa de Ciências do Desporto, 9(1), 81-89.         [ Links ]

13. Hopkins, WG. (2000). Reliability for consecutive pair of trials (Excel Spreadsheet). In: A new view of Statistics. Sportsci.org: Internet Society for Sport Science. Obtained from: sportsci.org/resource/stats/xrely.xls.         [ Links ]

14. Hopkins, WG. (2000a). Measures of Reliability in Sports Medicine and Science. Sports Medicine. 30, 1-15.         [ Links ]

15. Hughes, M., & Franks, I. M. (2004). Notational analysis of sport: Systems for better coaching and performance in sport. London: Psychology Press.         [ Links ]

16. Hughes, M. D., & Bartlett, R. M. (2002). The use of performance indicators in performance analysis. Journal of Sports Sciences, 20(10), 739-754.         [ Links ]

17. Liu, H., Hopkins, W., Gómez, M. A., & Sampedro J. (2013). Inter-operator reliability of live football match statistics from OPTA Sportsdata. International Journal of Performance Analysis in Sport, 13(3), 803-821.         [ Links ]

18. O'Donoghue, P. (2007). Reliability issues in performance analysis. International Journal of Performance Analysis in Sport, 7(1), 35-48.         [ Links ]

19. O'Donoghue, P. (2010). Research methods for sports performance analysis. London: Routledge.         [ Links ]

20. O'Donoghue, P., & Holmes, L. (2015). Data Analysis in Sport. London; Routledge.         [ Links ]

21. Prieto, L., Lamarca, R., & Casado, A. (1998). La evaluación de la fiabilidad en las observaciones clínicas: el coeficiente de correlación intraclase. Medicina clínica, 110(4), 142-145.         [ Links ]

22. Reina Gómez, A., & Hernández Mendo, A. (2012). Revisión de indicadores de rendimiento en fútbol. Revista Iberoamericana de Ciencias de la Actividad Física y el Deporte, 1, 14.         [ Links ]

23. Ribeiro, P. (2008). Do modelo e concepção de jogo à análise da performance no futebol: o treino enquanto indutor da operacionalização de um modo de jogar específico: estudo de caso na equipa de sub-19 do Futebol Clube do Porto.(Thesis).         [ Links ]

24. Robinson, G., and O'Donoghue, P. (2007). A weighted kappa statistic for reliability testing in performance analysis of sport. International Journal of Performance Analysis in Sport. 7, 12-19.         [ Links ]

25. Sáez, A. T. (2007). Vocabulario de fútbol. Gijón: Trea.         [ Links ]

26. Sampedro, J. (1999). Fundamentos de táctica deportiva. Madrid. Gymnos.         [ Links ]

27. Sampedro, J. (2013). Algunas consideraciones sobre el método de investigación científica del deporte. Sportk, 2(1), 23-35.         [ Links ]

28. Smith, T. B., & Hopkins, W. G. (2011). Variability and predictability of finals times of elite rowers. Medicine and Science in Sports and Exercise, 43(11), 2155-2160.         [ Links ]

29. Viera, A. J., and Garrett, J. M. (2005). Understanding inter-observer agreement: the kappa statistic. Family.         [ Links ]

 

 

Correspondence:
Miguel Ángel Gómez Ruano.
Facultad de CC de la Actividad Física y del deporte.
INEF-Madrid.
Universidad Politécnica de Madrid.
C/ Martín Fierro, 7.
28040. Madrid (España).

Recibido: 05/05/2015
Aceptado: 02/02/2016