EQOL Journal (2021) 13(2): 13-20


Psychometric properties of Bull’s Mental Skills Questionnaire in a Turkish population

Bülent O. Miçooğullari1• David J. Edwards2 • Rıdvan Ekmekçi3

Received: 27th September, 2021

DOI: 10.31382/eqol.211202

Accepted: 1st November, 2021


© The Author(s) 2021. This article is published with open access.



The purpose of this research was to examine the reliability and validity of Bull’s Mental Skills

Questionnaire (BMSQ), measuring seven psychological skills resulting in a total scale score, within Turkey. The sample consisting of 163 males and 131 females, totaling 294 athletes, completed the 28 item BMSQ. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were undertaken. EFA results yielded a satisfactory seven-factor solution, the same as the English version. Cronbach alpha (α) reliability indices were as follows: concentration ability (0.71), anxiety and worry management (0.63), relaxation ability (0.69), imagery ability (0.73), self-confidence (0.72), motivation (0.64) and mental preparation (0.52). The BMSQ explained 62% of the total variance. Moreover, model fit indices calculated during CFA for the 28 item and 7 sub-dimensions model of BMSQ indicated an acceptable fit [Chi-Square (ꭓ²)=541.2, df=328, ꭓ²/df=1.65, Root Mean Square Error Approximation (RMSEA)=0.05, Comparative Fit Index (CFI)=0.95, Goodness of Fit Index (GFI)=0.90, Normed Fit Index (NFI)=0.89, and Non Normed Fit Index (NNFI)=0.90]. Thus, the original


1Nevşehir Hacı Bektaş Veli University, Physical Education and Sport Department, Nevşehir,


2University of Zululand, Psychology Department, KwaDlangezwa, South Africa

3Pamukkale University, Sport Science Faculty, Sports Management Department, Denizli, Turkey

7-factor solution was supported with the data collected from Turkish participants. Results demonstrated that the BMSQ is a valid and reliable instrument for the Turkish population.

Keywords Bull’s Questionnaire • Exploratory and Confirmatory Factor Analysis • Turkish Population.


Sport as a concept is associated with several disciplines including psychology. Sport and exercise can have an impact on the mental state of sports people. Mental skills (MS) are naturally occurring, daily utilized, improvable abilities and/or techniques. They are interrelated and form a unique, composite, and inseparable whole (Weinberg & Gould, 2014).

Sport success is partly determined by physical conditioning, skills and preparation. It is, nevertheless, also influenced by psychological factors such as motivation, emotional control and self-confidence, which are considered to be mental skills (Smith et al., 2007). MS can be enhanced through mental skills training (MST) programs. MST can be used to overcome problems that can negatively affect athletes’ performance (e.g. stress due to hostile opposition supporters or unexpected weather conditions). The MST that athletes do prior to games can optimize their performance and help them achieve their goals. In general, MST programs are combined package training techniques used to develop a variety of naturally occurring psychological skills (Vealey, 2007). Initial MST interventions were focused on and trained single skills such as mental imagery, atte-


EQOL Journal (2021) 13(2): 13-20

ntion control, goal setting or motivation (Tenenbaum

&Eklund, 2007). Broad-based MST programs have special value for athletes as they include biological, social and spiritual components, which can improve general life skills and development (Sanchez & Lesyk, 2001).

The accurate assessment of mental skills is crucial for the development of health and sport in general and for the field of sport and exercise psychology in particular (Moran, 1993). As holistic, overlapping, naturally occurring, daily utilized, learned abilities, MS are interrelated and form a unique, composite, inseparable whole (Bull et al., 1996; Weinberg & Gould, 2014). They can be conceptually divided into distinct, but arbitrary, categories for purposes of research, teaching, learning and training. Initial MS studies focused on measuring individual MS and few studies measured MS collectively using psychometric instruments by establishing local and/or international norms. Such studies were initially conducted in the USA, Europe and Australia and researchers developed various scales to measure athletes’ mental skills.

In 1987, Mahoney and colleagues developed the Psychological Skill Scale (PSS) which has been used in various studies. The scale contained 51 true-false questions that aim to identify and evaluate the psychological skills linked to the performance of successful athletes. The scale was later revised to a newer version with 45 items and 6 sub-sections. The subscales that emerged were anxiety control, trust, concentration, motivation, team integrity and mental preparation. This scale was successful in revealing and classifying gender differences (White, 1993). However, researchers encountered a number of important statistical problems such as low alpha coefficients (Chartrand et al., 1992), invalid factor structure (Tammen & Murphy, 1990) and low fit indices in confirmatory factor analysis results.

Following this, another MS assessment tool, the Athletic Coping Skill Inventory (ACSI)-28 aiming to measure the mental skills utilized in the process of performance improvement and stabilization was developed by Smith et al. (1995). The completion of the inventory lasted five years (1990 – 1995). The development process of the initial version of ACSI had started with 42 questions and 8 sub-dimensions and, during the five years’ development period, the number of items decreased to 28 questions and sub- dimensions to 7. Those sub-dimensions were; coping with adversity, goal setting/mental preparation,

concentration, confidence and achievement


motivation, coachability, freedom from worry and ability to cope with difficulties (Smith & Christensen, 1995). The Turkish adaptation of ACSI-28 was completed by Özcan and Günay (2017). Their sample consisted of 318 students (165 girls, 153 boys) in high schools. A test-retest technique was used in this study and compatible indexes of the model were found to be χ2 = 924.74, χ2/sd = 3.33, RMSEA= 0.076, CFI= 0.89, Incremental Fit Index (IFI) = 0.90, NFI= 0.89 and NNFI= 0.90. Pearson correlation coefficients among the 7 factors from first and second application were calculated by the researchers for test-retest reliability. Test-retest reliability coefficient was between 0.62 and 0.83 (coping with adversity 0.72, goal setting/mental preparation 0.71, concentration 0.62, confidence and achievement motivation 0.81, coachability 0.83, freedom from worry 0.79 ability to cope with difficulties 0.83). Cronbach Alpha coefficients were between 0.62 and 0.80, which was similar to the values acquired by Smith et al. (1995).

Another MS measurement tool, the Ottawa Mental Skills Assessment Tools (OMSAT-3) has been developed by Durand-Bush, Salmela and Green- Demers (2001). The OMSAT-3 is a psychometrically sound measure that includes 48 items and 12 mental skill scales grouped under three broader conceptual components: (1) foundation skills, (2) psychosomatic skills and (3) cognitive skills. CFA results revealed that the OMSAT-3 scales have a sound factorial structure (Durand Bush et al., 2001). In terms of reliability, the OMSAT-3 scales mostly reflected satisfactory internal consistency, with values ranging between .68 and .88. Previous administrations of the OMSAT-3 have found that it is useful for generating discussion, confirming current training and competitive behaviors and determining what is beneficial for each athlete. The validity and reliability study of the Turkish version of the scale was studied by Erhan et al., (2015). Their study was conducted with the participation of 220 (92 women, 128 men) volunteer athletes between the ages of 18 and 33. The compatibility index values for the Turkish version of the scale was found to be RMSEA=0.072,

Standardized Root-Mean-Square Residual (SRMR)=0.066, NFI=0.92, NNFI=0.95 and CFI=0.956. The internal consistency of the scale was calculated with Cronbach’s Alpha the results of which ranged between 0.55 (fear control) and 0.75 (struggle plan). The general Cronbach Alpha coefficient of the scale was 0.94.

In terms of evaluating MS collectively, few such questionnaires are as comprehensively composed as Bull’s Mental Skills Questionnaire that consists of 28

EQOL Journal (2021) 13(2): 13-20

questions that evaluate participants using a six-point Likert scale (Bull et al., 1996; Snauwert, 2001). Despite being developed in the UK, this measurement tool was not designed specific for the UK context. The researchers who developed this tool aimed for its use in international contexts. As such, it has been trialled and validated successfully in the South Africa context (Edwards & Edwards, 2012). The researchers who developed this tool aimed for it to be used in international contexts. As such, it has more recently been trialed and validated in South African and United Kingdom university contexts (Buscombe et al., 2011; Edwards & Steyn, 2011; Edwards et al., 2011).

The aim of this research was to adapt Bull’s Mental Skill Questionnaire for use in the Turkish context, and to evaluate its validity and reliability.



The study sample consisted of 163 males (mean±SD: age 22.12±3.22 years) and 131 females (mean±SD: age 21.42 ± 3.85 years) totaling 294 athletes (mean±SD: age 21.25±4.87 years) who voluntarily participated. Participant athletes had different levels of competition experience. In addition, they had different sport backgrounds (12 different sports – 6 team sports and 6 individual sports – including handball, track and field, basketball, volleyball, soccer and badminton).

Ethical administrative procedures

Prior to data collection, approval was obtained from the Ethics Committee of Nevşehir Hacı Bektaş Veli University with the 2018.10.110 ethical clearance number being granted. The author of the original scale was contacted to ask for permission to use the scale, which he kindly gave. During the questionnaire administration, each participant was briefed with regards to what is involved in the study and how their responses would be kept confidential. After participant briefing, athletes were informed that participation was voluntary and that they could withdraw at any time during the data collection process. After this, data was only collected from athletes who gave consent and volunteered to participate.

Measuring instrument

Bull’s Mental Skills Questionnaire (BMSQ), which consists of 28 items, was developed in the UK with

the aim of measuring a number of mental skills that are commonly used in sports by athletes. These skills, also the sub-dimensions of the questionnaire, are; imagery ability (IA) reflected in items 1 to 4, mental preparation (MP) reflected in items 5 to 8, self- confidence (SC) reflected in items 9 to 12, anxiety and worry management (AW) reflected in items 13 to 16, concentration ability (CA) reflected in items 17 to 20, relaxation ability (RA) reflected in items 21 to 24 and motivation (M) reflected in items 25 to 28 (Bull et al., 1996). Individuals who completed the BMSQ questionnaire were provided with a 6-point Likert scale and were asked to choose one of the six options provided to them (1 representing “strongly disagree” and 6 representing “strongly agree”).


The Turkish version of the BMSQ was only completed by individuals who voluntarily participated and signed consent forms. The translation procedure of the BMSQ was undertaken by faculty members from English Language Department of Nevşehir Hacı Bektaş Veli University. They forward-translated the English version into Turkish and the translated versions of the BMSQ were compared for deviations. Then another bilingual language expert was asked to back-translate the Turkish version into English to examine the meaning retention of the items. Three academics who are experts in the areas of physical education and sports sciences, sports psychology and psychometrics, reviewed the content of the preliminary BMSQ version to ensure that the questions were culturally appropriate to the Turkish population.

After this translation protocol, the final version of BMSQ in Turkish was ready and a cross-sectional design was used in this study. Data collection process took place between March and April 2018. In total, 332 individuals completed the questionnaire, 38 of these were excluded from the study due to being incomplete, leaving 294 responses available for statistical analyses.

Data analysis

Results of Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) are presented in the following section. To determine the factor structure of the multidimensional questionnaire, EFA was performed using the Statistical Package for the Social Sciences (SPSS) 21.0. Following this, CFA was conducted using Analysis Moments of Structures (AMOS) 18. Comparative fit index (CFI>0.90, acceptable), non-normed fit index (NNFI>0.90


EQOL Journal (2021) 13(2): 13-20

acceptable), normed fit index (NFI>0.90) (Jaccard & Wann, 1996) and root mean square error of approximation (RMSEA<0.08, adequate model fit; Öcal, 2011) were used to as criteria to check the model fit (Maruyama, 1998). The measurement model based on the fit indices was evaluated for construct validity. And, internal consistency of the adapted scale was established by computation of Cronbach’s coefficient alpha.


Initially, the factorability of the BMSQ (28 items) was examined. Several well-recognized criteria for the factorability of a correlation were used. Firstly, it was observed that 24 of the 28 items correlated at .3 or above with at least one other item, suggesting reasonable factorability. Secondly, the Kaiser-Meyer- Olkin measure of sampling adequacy was .81, above the commonly recommended value of .6, and Bartlett’s test of sphericity was significant (χ2 (378)

=2218.13, p<0.05). The diagonals of the anti-image correlation matrix were also all over .5 and lastly, the

commonalities were all above .3, further confirming that each item shared some common variance with other items. Given these overall indicators, factor analysis was deemed to be suitable for all 28 items.

For the final stage, a principal components factor analysis of the BMSQ items, using varimax and oblimin rotations, was conducted, with seven factors explaining 54% of the variance. An oblimin rotation provided the best-defined factor structure. The factor loading matrix for this final solution is presented in Table 2. The factor labels proposed by Bull et al. (1996) suited the extracted factors. Internal consistency for each of the scales was examined using Cronbach’s alpha. The alpha levels were moderate: concentration ability (4 items; 0.71), anxiety and worry management (4 items; 0.63), relaxation ability (4 items; 0.69), imagery ability (4 items; 0.73), self- confidence (4 items; 0.72), motivation (4 items; 0.64) and mental preparation (4 items; 0.52). No substantial increases in alpha for any of the scales could have been achieved by eliminating any item. In the end, Turkish adaptation of BMSQ explained 62% of the total variance.

Table 1. Descriptive statistics for the seven bull’s mental skills questionnaire sub-dimensions (n=294)


No. of


















Imagery ability






Mental preparation






Self Confidence






Anxiety and worry management






Concentration ability






Relaxation ability


















Table 2. Factor loadings based on a principal components analysis with oblimin rotation for 28 items from the BMSQ (n=294)









Item1 (IA)








Item2 (IA)








Item3 (IA)








Item4 (IA)








Item5 (MP)








Item6 (MP)








Item7 (MP)








Item8 (MP)








Item9 (SC)








Item10 (SC)








Item11 (SC)

















EQOL Journal (2021) 13(2): 13-20

Table 2 (continued). Factor loadings based on a principal components analysis with oblimin rotation for 28 items from the BMSQ (n=294)









Item12 (SC)








Item13 (AWM)








Item14 (AWM)








Item15 (AWM)








Item16 (AWM)








Item17 (CA)








Item18 (CA)








Item19 (CA)








Item20 (CA)








Item21 (RA)








Item22 (RA)








Item23 (RA)








Item24 (RA)








Item25 (M)








Item26 (M)








Item27 (M)








Item28 (M)








*Note: Imagery ability (IA); Mental preparation (MP); Self-confidence (SC); Anxiety and worry management (AWM); Concentration ability (CA); Relaxation ability (RA); Motivation (M)

After EFA, CFA was used to test the factor structure that shows the sub-dimensions of Bull’s Mental Skills Questionnaire over the data gathered from Turkish athletes. For a model with 7 factors (imagery ability, mental preparation, self-confidence, anxiety and worry management, concentration ability, relaxation ability and motivation) set in the

original sub-dimension, goodness of fit statistics was calculated. The results were as follows, χ2=504.01 (df=278, p=0.00), χ2 504/ df=1.81, RMSEA=0.05, CFI=0.95, NNFI=0.94, NFI=0.90. Therefore, it is evident that the model is coherent at a satisfactory level

Figure 1. Hypothesized model of BMSQ


EQOL Journal (2021) 13(2): 13-20


The goal of the study was to evaluate the psychometric properties of the Bull’s Mental Skills Questionnaire (Bull et al., 1996) on a Turkish population. In this study, exploratory and confirmatory factor analyses were used to analyze the data collected from 294 athletes from 6 team sports and 6 individual sports. EFA and CFA results supported the initial structure of the questionnaire for the overall model.

The original BMSQ was based on Nelson and Hardy’s (1990) Psychological Skills in Sport Questionnaire (PSSQ), which consists of the following categories: imagery skill, mental preparation, self-efficacy, cognitive anxiety, concentration skill, relaxation skill and motivation.

A number of measurement tools that have been developed to evaluate mental toughness in sports are available in the literature. As explained earlier, such measurements tools (e.g., the PSS, the PSSQ, the ACSI and the OMSAT-3) have been trialed in different environments. Nevertheless, all of the tools developed have continuously failed to demonstrate high levels of validity and reliability that was expected from them (Bull et al., 1996). For instance, even though results of several studies displayed meaningful factorial validity scores, they failed in establishing satisfactory face or external validity. In addition, there have been occasions in which satisfactory validity was established. However, when that happened, researchers failed in explaining why certain items were included in a sub-domain or not due to the development process not being grounded on theory. The BMSQ was chosen as the instrument to be used in the present research study for the above reasons as well as the convenience of its items, its’ empirical application, reliability, and validity.

Consistent with our findings, Cronbach’s Alpha reliability values were reported above 0.78 in the original study the questionnaire was developed. Similarly, Snauwaert (2001) analyzed psychometric properties of a Dutch version of BMSQ and statistical results indicated that the BMSQ had acceptable and generally high levels internal consistency (IA=.80 MP=0.64; SC=0.62; AWM=0.61; CA=0.59; CA=0.72; M=0.72. Although, in the validation of the BMSQ in an Afrikaans speaking population, the self- confidence subscale of the BMSQ was removed due to the EFA results, in this study Cronbach alpha values were reported as considerably high as follows; CA (0.83), AWM (0.81), RA (0.74), IA (0.61), M


(0.75) and MP (0.57) (Kruger et al., 2015). In summary, similar to the current findings, other validity and reliability studies demonstrated that BMSQ is a valid and reliable data collection tool in different cultures.


Concrete evaluation of mental skills is crucial for the development of psychological health and sports in general. As holistic, super-imposed, naturally occurring, daily used and learned skills; mental skills are interdependent and form a single, compound and inseparable whole. The present results of the Bull’s Mental Skills Questionnaire-Turkish version suggest the adapted version has relatively strong psychometric properties, and is a valid and reliable test instrument to evaluate mental skills levels in Turkish sports contexts (athletes, university students, etc.). The present study has some limitations. The participants were Turkish university students who were amateur athletes competing in a number of team and individual sports. The questionnaire, therefore, may not be suitable for use with professional level athletes in a high-level sport environment. Thus, continued evaluation of the BMSQ is necessary. Moreover, additional types of invariance testing (e.g., temporal, cross-cultural, and sport type), as well as other ongoing construct validity evaluation, needs to be considered in future research to gather new evidence on mental skills.


Gratitude is expressed to Nina Carmen Robinson for research assistance. This work is based on research supported by the University of Zululand and the South African National Research Foundation (NRF). Any opinion, finding and conclusion or recommendation expressed in this material is that of the author(s) and the NRF does not accept any liability in regard thereto.


Bull, S., Albinson, J. & Shambrook, C. (1996). The mental game plan. Getting psyched for sport. Eastbourne, East Sussex: Sports Dynamics.

Buscombe, R.M., Edwards, D.J. & Steyn, B.J.M. (2011). Use of and relationships between mental skills in South African and United Kingdom university students.

EQOL Journal (2021) 13(2): 13-20

Poster presented at the 16th Annual Congress of the European College of Sport Science, Liverpool, United Kingdom.

Chartrand, J.M.; Jowdy, D.P. & Danish, S.J. (1992). The

Psychological Skills Inventory for Sports: Psychometric characteristics and applied implications. Journal of Sport and Exercise Psychology, 14, 405- 413.

Durand-BUSH, N.; Salmela, J.H. & Green-Demers, I. (2001). The Ottawa Mental Skills Assessment Tool (OMSAT-3). The Sport Psychologist, 15(1), 18-19.

Edwards, S.D. & Edwards, D.J. (2012). The description and evaluation of a breath based psychological skills training program for health and sport. African Journal for Physical Health Education, Recreation and Dance, 13(4), 380-399.

Edwards, D.J. & Steyn, B.J.M. (2011). The establishment of norms for the Bull’s mental skills questionnaire in South African university students: An exploratory study. African Journal for Physical Health Education, Recreation and Dance, 17(3), 526-534.

Edwards, D.J., Steyn, B.J.M. & Buscombe, R.M. (2011). Mental skills in South African undergraduate university students: An exploratory study. Poster presented at the 13th European Congress of Sport Psychology, Madeira, Portugal.

Erhan S.E.; Güler, M. Ş.; Ağduman, F. & Gerek Z. (2015). Ottawa Mental Ability Assessment Scale-3 Turkish validity and reliability study. Journal of Kafkas Education Researches, 2(2), 33-39.

Jaccard, J. & Wann, C. K. (1996). LISREL approaches to interaction effects in multiple regression. Thousand Oaks, CA: Sage Publications.

Kruger, A.; Edwards, D.J. & Edwards, S.D. (2015). Bull’s Mental Skills Questionnaire validation in an Afrikaans speaking population. African Journal for Physical, Health Education, Recreation and Dance, 21(2), 612- 618.

Maruyama, G. M. (1998). Basics of structural equation modeling. Thousand Oaks, Ca: Sage.

Moran, A. (1993). Conceptual and methodological issues in the measurement of mental imagery skills in athletes. Journal of Sport Behavior, 16(3), 156.

Nelson, D. & Hardy, L. (1990). The development of an empirically validated tool for measuring psychological skill in sport. Journal of Sport Science, 8(10), 71.

Öcal, K., (2011). Predicting employee performance in non- profit sport organizations: The role of managerial and financial performance and the mediating role of

support for innovation and individual creativity. Unpublished doctoral dissertation. Middle East Technical University, Turkey.

Özcan, V. & Günay, M. (2017). The Turkish adaptation of Athletic Coping Skills Inventory-28 (ACSI-28): The validity and reliability study. Turkish Journal of Sport and Exercise, 19(1), 130-136.

Sanchez, X. & Lesyk, J.L. (2001). Mental skills training using the ‘nine mental skills of successful athletes’ model. Proceedings of the 10th World Congress of Sport Psychology, Volume 4, 85-86. Thessaloniki: Christodoulidi.

Smith, R.E.; Smoll, F.L. & Cumming, S.P. (2007). Effects of a motivational climate intervention for coaches on children’s sport performance anxiety. Journal of Sport & Exercise Psychology, 29, 39-59.

Smith, R.E.; Schitz, R.W.; Smoll, F.L. & Ptacek, J.T. (1995). Development and validation of a

multidimensional measure of sport-specific psychological skills: The athletic coping skills inventory-28. Journal of Sport & Exercise Psychology, 17, 379-398.

Smith, R.E. & Christensen, D.S. (1995). Psychological skills as predictors of performance and survival in professional baseball. Journal of Sport and Exercise Psychology, 17, 399-415.

Snauwaert, E. (2001). A psychometric evaluation of Bull’s mental skills questionnaire: A study on Flemish athletes. Presented at the 10 World Congress of Sport Psychology, Skiathos, Greece.

Tammen, V.V. & Murphy, S.M. (1990). Reevaluating the Psychological Skills Inventory for Sports: Factor analysis and implications. Communication to the Annual Meeting of the North American Society of Psychology of Sport and Physical Activity, Asilomar, CA.

Tenenbaum, G. & Eklund, R. C. (2007). Handbook of sport psychology (3rd ed.). New York: John Wiley.

Vealey, S.R. (2007). Mental Skills Training in Sport. In G. Tenenbaum, R. Eklund and R. Singer (Eds.), Handbook of sport psychology. New Jersey: Wiley.

Weinberg, R.S. & Gould, D. (2014). Foundations of Sport and Exercise Psychology (6th ed.). Champaign, IL: Human Kinetics.

White, S.A. (1993). The relationship between psychological skills, experience, and practice commitment among collegiate male and female skiers. The Sport Psychologist, 7, 49-57.


EQOL Journal (2021) 13(2): 13-20

How to cite this article:

Miçooğullari, B. O., Edwards, D. J., & Ekmekçi, R. (2021). Psychometric properties

APA:of Bull’s Mental Skills Questionnaire in a Turkish population. Exercise and Quality of Life, 13(2), 13-20. doi:10.31382/eqol.211202

Miçooğullari, Bülent O. , David J. Edwards and Rıdvan Ekmekçi. "Psychometric

MLA:properties of Bull’s Mental Skills Questionnaire in a Turkish population" Exercise and Quality of Life 13.2 (2021): 13-20.

Miçooğullari, Bülent O. , David J. Edwards, and Rıdvan Ekmekçi. "Psychometric

Chicago:properties of Bull’s Mental Skills Questionnaire in a Turkish population" Exercise and Quality of Life 13, no. 2 (2021): 13-20.