The content validity requires to assess not only the clarity and relevance of the items, but also the substantial content
Main Article Content
Abstract
The aim of the study was to use the substantive validity strategy to evaluate the content of the UNIPSICO’s workload scale. 30 workers from Lima (Perú) with undergraduate or graduate studies and with one year of work experience participated. Participants judged the correspondence of the workload items with their construct, and it was also contrasted with other constructs (role conflict, coping and labor self-efficacy); additionally, the clarity of the items was evaluated. High levels of item’s clarity was found; and substantive validity of all items was satisfactory to evaluate the divergence with other constructs. We discuss the importance of including non-expert participants in content validity methods, conceptual differentiation of items, and in constructs associated with psychosocial factors.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
La RACC aplicará la licencia internacional de atribuciones comunes creativas (Reconocimiento 4.0 Internacional: https://creativecommons.org/licenses/by/4.0/).
Bajo esta licencia, se permite cualquier explotación de la obra, incluyendo la explotación con fines comerciales y la creación de obras derivadas, la distribución de las cuales también está permitida sin ninguna restricción. Esta licencia es una licencia libre según la Freedom Defined. La única condición es que siempre y en todos los casos se cite a los autores y a la fuente original de publicación (i.e., RACC). Esta licencia fue desarrollada para facilitar el acceso abierto, gratuito y libre a trabajos originales científicos y artísticos.
How to Cite
References
Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(2), 955–959. doi: 10.1177/001316448004000419
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(2), 131-142. doi: 10.1177/0013164485451012
Anderson, J. C., & Gerbing, D. W. (1991). Predicting the performance of measures in a confirmatory factor analysis with a pretest assessment of their substantive validities. Journal of Applied Psychology, 76(3), 732-740. doi: 10.1037/0021-9010.76.5.732
Aravamudhan, N. R., & Krishnaveni, R. (2015). Establishing and reporting content validity evidence of new Training and Development Capacity Building Scale (TDCBS). Management, 20(1), 131–58
Arquer, I., Daza, F., & Nogareda., C. (1995). NTP 338: Ambigüedad y conflicto de rol. Barcelona: INSHT.
Bacharach, S. B., Bamberger, P., & Conley, S. C. (1990). Work processes, role conflict, and role overload: The case of nurses and engineers in the public sector. Work and Occupations, 17(2), 199-228. doi: 10.1177/0730888490017002004
Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186-3191. doi: 10.1097/00007632-200012150-00014
Byrne, B. (1994). Burnout: testing for the validity replication and invariance of causal structure across elementary, intermediate and secondary teachers. American Educational Research Journal, 31(3), 654-673. doi: 10.3102/00028312031003645
Calderón-De la Cruz, G., Merino-Soto, C., Juárez-García, A., & Jimenez-Clavijo, M. (2018). Validación de la escala de carga de trabajo en trabajadores peruanos. Archivos de Prevención de Riesgos Laborales, 21(3), 123-127. doi: 10.12961/aprl.2018.21.03.2
Chen, G., Gully, S. M., & Eden, D. (2001). Validation of a new general self-efficacy scale. Organizational Research Methods 4(1), 62-83. doi: 10.1177/109442810141004
Choi, B., & Juárez-García, A. (2017). Language issues in standard questionnaires for assessing psychosocial working conditions: the case of the JCQ and the ERIQ. En S. Cassilde, & A. Gilson (Eds.), Psychosocial Health at Work and Language. International Perspectives toward their Categorizations at Work (pp. 3-18). Cham, Switzerland: Springer. doi: 10.1007/978-3-319-50545-9_1
DeWalt, D. A., Rothrock, N., Yount, S., & Stone, A. A. (2007). Evaluation of item candidates: The PROMIS qualitative item review. Medical Care, 45(5), 12-21. doi: 10.1097/01.mlr.0000254567.79743.e2
Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: the consequences of using flawed test items on achievement examinations in medical education. Advances in Health Sciences Education: Theory and Practice, 10(2), 133-143. doi: 10.1007/s10459-004-4019-5
Farrel, A. M., Souchon, A. L., & Durden, G. R. (diciembre, 2003). The service leadership scale: a substantive validity test. Paper presentado en the Australian and New Zealand Marketing Academy Conference (ANZMAC), Melbourne, Australia.
Fiske, D. W. (1967). The subjects react to test. American Psychologist, 22(2), 287-296. doi: 10.1037/h0024523
Frone, M. R., Yardley, J. K., & Markel, K. (1997). Developing and testing an integrative model of the work-family interface. Journal of Vocational Behavior, 50(2), 145-167. doi: 10.1006/jvbe.1996.1577
Geurts, S. A. E., & Demerouti, E. (2003). Work/non-work interface: A review of theories and findings. En M. J. Schabracq, J. A. M. Winnubst, & C. L. Cooper (Eds.), The handbook of work and health psychology (pp. 279-312). Chichester, UK: John Wiley & Sons.
Gil-Monte, P. (2012). Riesgos psicosociales en el trabajo y salud ocupacional. Revista Peruana de Medicina Experimental y Salud Pública, 29(2), 237-241. doi: 10.1590/S1726-46342012000200012
Gil-Monte, P. R. (2014). Manual de psicosociología aplicada al trabajo y a la prevención de los riesgos laborales. Madrid: Pirámide.
Gil-Monte, P. R. (2016). La Batería UNIPSICO: propiedades psicométricas de las escalas que evalúan los factores psicosociales de demanda. Archivos de Prevención de Riesgos Laborales, 19(2), 86-94. doi: 10.12961/aprl.2016.19.02.2
Gil-Monte, P. R., & Peiró, J. M. (1997). Desgaste psíquico en el trabajo: el síndrome de quemarse. Madrid: Síntesis.
Gil-Monte, P., & García-Juesas, J. (2008). Efectos de la sobrecarga laboral y la autoeficacia sobre el síndrome de quemarse por el trabajo (burnout). Un estudio longitudinal en enfermería. Revista Mexicana de Psicología, 25(2), 329-337.
Gil-Monte, P., García-Juesas, J., & Caro, M. (2008). Influencia de la sobrecarga laboral y la autoeficacia sobre el síndrome de quemarse por el trabajo (burnout) en profesionales de enfermería. Revista Interamericana de Psicología, 42(2), 113-118.
Gil-Monte, P., López-Vílchez, J., Llorca-Rubio, J., & Sánchez, J. (2016). Prevalencia de riesgos psicosociales en personal de la administración de justicia de la comunidad valenciana (España). Liberabit, 22(1), 7-19.
Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing and Health, 20(3), 269–274. doi: 10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G
Greenglass, E. R., Burke, R. J., & Moore, K. A. (2003). Reactions to increased workload: effects on professional efficacy of nurses. Applied Psychology: An International Review, 52(2), 580-597. doi: 10.1111/1464-0597.00152
Hanley, K., Howard, M. C., Zhong, B., Soto, J. A., Pérez, C. R., Lee, E. A., … Minnick, M. R. (2015). The communication anxiety regulation scale: development and initial validation. Communication Quarterly, 63(1), 23-43. doi: 10.1080/01463373.2014.965836
Helm, S., Eggert, A., & Garnefeld, I. (2010). Modeling the impact of corporate reputation on customer satisfaction and loyalty using partial least squares. En V. Esposito, W. W. Chin, J. Henseler, & H. Wang (Eds.), Handbook of Partial Least Squares: Concepts, Methods and Applications (pp. 515-534). Berlin: Springer.
Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach to content validation. Organizational Research Methods, 2(2), 175-186. doi: 10.1177/109442819922004
Hoehle, H., Aljafari, R., & Venkatesh, V. (2016). Leveraging Microsoft's mobile usability guidelines: Conceptualizing and developing scales for mobile application usability. International Human-Computer Studies, 89(2), 35–53. doi: 10.1016/j.ijhcs.2016.02.001
Juárez-García, A. (2007). Factores psicosociales laborales relacionados con la tensión arterial y síntomas cardiovasculares en personal de enfermería en México. Salud Pública de México, 49(2), 109-117.
Kanai, A., & Wakabayashi, M. (2001). Workaholism among Japanese blue-collar employees. International Journal of Stress Management, 8(2), 129-145. doi: 10.1023/A:1009529314121
Kirch, W. (2008). Encyclopedia of Public Health. New York: Springer.
Lasch, K. E., Marquis, P., Vigneux, M., Abetz, L., Arnould, B., Bayliss, M., … Rosa, K. (2010). PRO development: rigorous qualitative research as the crucial foundation. Quality of Life Research, 19(8), 1087-1096. doi: 10.1007/s11136-010-9677-6
Lazarus, R. S., & Folkman, S. (1986). Estrés y procesos cognitivos. Barcelona: Martínez Roca.
Loevinger, J. (1957). Objective test as instruments of psychological theory. Psychology Reports, 3(2), 635-694. doi: 10.2466/pr0.1957.3.3.635
Maffei, L., Spontón, C., Spontón, M., Castellano, E., & Medrano, L. A. (2012). Adaptación del Cuestionario de Autoeficacia Profesional (AU-10) a la población de trabajadores cordobeses. Pensamiento Psicológico, 10(1), 51-62.
Magasi, S., Ryan, G., Revicki, D., Lenderking, W., Hays, R. D., Brod, M., … Cella, D. (2012). Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting. Quality of Life Research, 21(5), 739-746. doi: 10.1007/s11136-011-9990-8
Manson, S. (1997). Cross-cultural and multi-ethnic assessment of trauma. En J. P. Wilson, & T. M. Keane (Eds.), Assessing psychological trauma and PTSD: A handbook for practitioners (pp. 239-266). New York: Guilford.
May, L. A., & Warren, S. (2001). Measuring quality of life of persons with spinal cord injury: Substantive and structural validation. Quality of Life Research, 10(2), 503-515. doi: 10.1023/A:1013027520429
Meliá, J. L., Nogareda, C., Lahera, M., Duro, A., Peiró, J. M., Salanova, M., & Gracia, D. (2006). Principios comunes para la evaluación de riesgos psicosociales en la empresa. En J. L. Meliá, C. Nogareda, M. Lahera, A. Duro, J. M. Peiró, R. Pou, … F. Martínez-Losa (Eds.), Perspectivas de Intervención en Riesgos Psicosociales. Evaluación de Riesgos (pp. 13-36). Barcelona: Foment del Treball Nacional.
Merino-Soto, C. (2016). Percepción de la claridad de los ítems: Comparación del juicio de estudiantes y jueces-expertos. Revista Latinoamericana de Ciencias Sociales, Niñez y Juventud, 14(2), 1469-1477. doi: 10.11600/1692715x.14239120615
Merino-Soto, C., & Livia, C. (2009). Intervalos de confianza asimétricos para el índice la validez de contenido: Un programa Visual Basic para la V de Aiken. Anales en Psicología, 25(1), 169-171.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. doi: 10.1037/0003-066X.50.9.741
Mokkink, L. B., Terwee, C. B., Knol, D. L., Stratford, P. W., Alonso, J., Patrick, D. L., ... De Vet, H. C. (2006). Protocol of the COSMIN study: Consensus-based Standards for the selection of health Measurement Instruments. BMC Medical Research Methodology, 6(2), 2-10. doi: 10.1186/1471-2288-6-2
Mokkink, L. B., Terwee, C. B., Knol, D. L., Stratford, P. W., Alonso, J., Patrick, D. L., ... De Vet, H. C. (2010). The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Medical Research Methodology, 10(2), 22. doi: 10.1186/1471-2288-10-22
Mora-Ríos, J., Bautista-Aguilar, N., Natera, G., & Duncan, P. (2013). Adaptación cultural de instrumentos de medida sobre estigma y enfermedad mental en la Ciudad de México. Salud Mental, 36(1), 9-18. doi: 10.17711/SM.0185-3325.2013.002
Morse, D. T., & Morse, L. W. (2002). Are undergraduate examiner’s perceptions of item difficulty related to item characteristics? Perceptual and Motor Skills, 95(3-2), 1281-1286. doi: 10.2466/pms.2002.95.3f.1281
Nevo, B. (1995). Examinee Feedback Questionnaire: Reliability and validity measures. Educational and Psychological Measurement, 55(3), 499-504. doi: 10.1177/0013164495055003017
Organización Internacional del Trabajo. (2016). Estrés en el Trabajo: un reto colectivo. (Informe No. 1). Ginebra: Organización Internacional del Trabajo.
Patlán, J. (2013). Efecto del burnout y la sobrecarga en la calidad de vida en el trabajo. Estudios Gerenciales, 29(129), 445-455. doi: 10.1016/j.estger.2013.11.010
Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E., & Ring, L. (2011). Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health, 14(8), 978-988. doi: 10.1016/j.jval.2011.06.013
Penfield, R. D., & Giacobbi, P. R. Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. doi: 10.1207/s15327841mpee0804_3
Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(2), 489-497. doi: 10.1002/nur.20147
Polit, D. F., Beck, C. T., & Owen, S. (2007). Is the CVI an acceptable indicator of content validity? Research in Nursing & Health, 30(2), 459-467. doi: 10.1002/nur.20199
Rothman, M., Burke, L., Erickson, P., Leidy, N. K., Patrick, D. L., & Petrie, C. D. (2009). Use of existing patient-reported outcome (PRO) instruments and their modification: the ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PRO Task Force Report. Value Health, 12(8), 1075-1083. doi: 10.1111/j.1524-4733.2009.00603.x
Schnall, P. L., Dobson, M., & Rosskam, E. (2009). Unhealthy work: Causes, consequences, cures. London: Baywood Publishing Company.
Shepard, K. F., Jensen, G. M., Schmoll, B. J., Hack, L. M., & Gwyer, J. (1993). Alternative approaches to research in physical therapy: Positivism and phenomenology. Physical Therapy, 73(2), 88-97. doi: 10.1093/ptj/73.2.88. PMID: 8421722.
Skinner, N., & Pocock, B. (2008). Work-life conflict: is work time or work overload more important? Asia Pacific Journal of Human Resources, 43(3), 303-315. doi: 10.1177/1038411108095761
Tan, S. H., & Tan, S. B. (2010). The correct interpretation of confidence intervals. Proceedings of Singapore Healthcare, 19(3), 276–278. doi: 10.1177/201010581001900316
Tovalín, H., & Rodríguez, M. (2013). Conceptos básicos en la evaluación del riesgo psicosocial en los centros de trabajo. En A. J. García, & A. Camacho Ávila (Eds.), Reflexiones teórico-conceptuales de lo psicosocial en el trabajo (pp. 95-112). Cuernavaca, México: Universidad Autónoma del Estado de Morelos, Juan Pablos Editor.
Unda, S., Uribe, F., Jurado, S., García, M., Tovalín, H., & Juárez, A. (2016). Elaboración de una escala para valorar los factores de riesgo psicosocial en el trabajo de profesores universitarios. Journal of Work and Organizational Psychology, 32(2), 67-74. doi: 10.1016/j.rpto.2016.04.004
Urrutia, M., Barrios, S., Gutiérrez, M., & Mayorga, M. (2014). Métodos óptimos para determinar la validez de contenido. Educación Médica Superior, 28(3), 547-558.
van Kooten, J. A. M. C., Terwee, C. B., Kaspers, G. J. L., Raphaёle, R. L., & van Litsenburg, R. R. L. (2016). Content validity of the Patient-Reported Outcomes Measurement Information System Sleep Disturbance and Sleep Related Impairment item banks in adolescents. Health and Quality of Life Outcomes, 14(2), 92-101. doi: 10.1186/s12955-016-0496-5
Veloutsou, C. A., & Panigyrakis, G. G. (2004). Consumer brands managers´ job stress, job satisfaction, perceived performance and intention to leave. Journal of Marketing Management, 20(2), 105-131. doi: 10.1362/026725704773041140
Wood, M. (2019). Simple methods for estimating confidence levels, or tentative probabilities, for hypotheses instead of p values. Methodological Innovations, 1(2), 1-9. doi: 0.1177/2059799119826518
Yao, G., Wu, C. H., & Yang, C. T. (2007). Examining the content validity of the WHOQOL- BREF from respondents’ perspective by quantitative methods. Social Indicator Research, 85(3), 483-498. doi: 10.1007/s11205-007-9112-8
Zabkar, V. (2000). Some methodological issues with structural equation modeling application in relationship quality context. New Approaches in Applied Statistics, 16(2), 211-224.