Servicios personalizados
Inglés (pdf)
Articulo en XML
Referencias del artículo
Traducción automática
Enviar articulo por email
Citado por SciELO 
Citado por Google 
Similares en
SciELO 
Similares en Google 
Permalink
Anyone who has read Raymond Carver's story What we talk about when we talk about love will remember a conversation in which the use of the term love is dangerously far from the common usage. Or perhaps not so far. In principle, it would suffice to compare what is narrated by the characters with a dictionary of authorities or with the definitions provided by experts. If we consider, with Wittgenstein (1953), that meaning is usage, there is the option of analyzing a linguistic corpus and contrasting the results with the official definitions mentioned above.
By analogy, if we want to find out what we talk about when we talk about psychometrics, there could be several complementary options, such as consulting the APA dictionary (American Psychological Association, n.d.), interviewing the presidents of scientific societies (Wijsen & Borsboom, 2021), or analyzing an updated corpus, such as that of Español del Siglo XXI [Spanish of the 21st Century] or CORPES XXI (Real Academia Española [Royal Spanish Academy], n.d.). Thus, we would be dealing with the meaning of terms from different perspectives: the first two are of the reflexive type, provided by international experts; the third includes, in addition, the pre-reflexive point of view, that of ordinary usage. Philosophers, psychologists, and neuroscientists have been dealing with the latter, the one related to what is known as seeing meanings (Lieberman, 2022; Morris, 2019; Wittgenstein, 1953).
We can start with a general definition. Psychometrics is the branch of psychology concerned with the quantification and measurement of mental attributes, behavior, performance, and the like, as well as with the design, analysis, and improvement of tests, questionnaires, and other instruments used in such measurement (American Psychological Association, n.d.). It is clear that it refers to the psychological field (although, if we were to look for the term psychometry, synonymous with psychometrics, we would also find a meaning related to parapsychological ability with the caveat−as if it were necessary−that there has been no verification of such an aptitude).
One might expect that the meaning would be the same for experts reflecting on psychometrics. As part of her doctoral work, Lisa Wijsen interviewed 20 scientists who had been presidents of the Psychometric Society (Wijsen & Borsboom, 2021) and found a surprising variety of interests, outside psychology in most cases. In the analysis by the cited authors, five types emerged: (1) psychologist: always has a substantive interest, aims to improve psychological understanding; (2) consultant: prefers to help psychologists solve methodological and statistical problems; (3) data analyst: aims to make predictions or summarize data; (4) engineer: interested in building technologically advanced artifacts with a clear application in society (innovative types of tests, such as computer adaptive testing or simulation assessments, but also software programs); (5) mathematician: knowledge for knowledge's sake is enough. Very few considered psychology to be consubstantial with psychometric work. Psychometrics, as understood by most of these experts, does not necessarily have to take the psychological into consideration; the distancing between the two is obvious here. Klaas Sijtsma, one of the few chairmen who considered psychometrics to be united with psychology, had already warned about this distancing, stating that there is no compelling reason to consider the mathematical structure of a latent trait model or any other psychometric model to be coincident with the theoretical structure of an attribute (Sijtsma, 2012).
This variety in the interpretation of meaning gives rise to the problem of the jingle fallacy (groups of entities that can be distinguished according to their attributes are assumed to be the same because they have the same name), well known in psychometric fields for its impact on construct validity. Recently, a committee of scientists from the U.S. Academies of Sciences, Engineering, and Medicine has drawn attention to the problems for the behavioral sciences arising from this and other fallacies related to meaning. Having unified definitions of the concepts that are important in each domain facilitates scientific communication, clinical application, and dissemination; moreover, it is considered key to accelerating progress in the behavioral sciences (National Academies of Sciences, Engineering, and Medicine, 2022). Thus, we consider that analyzing short texts from an updated corpus of Español del Siglo XXI [Spanish of the 21st Century] and contrasting them with the aforementioned meanings is an objective of both academic and professional interest in order to understand what we talk about when we talk about psychometrics (in Spanish).
Method
Procedure and Materials
We searched for psychometr* on October 10, 2022 in CORPES XXI (Real Academia Española [Royal Spanish Academy], 2022) which, in its latest version, version 0.94, contains some 350 million orthographic forms from Spain and Latin America from written texts and oral transcriptions, allowing us to analyze the use of a term in the main Spanish-speaking areas: Spain, Mexico-Central America, Río de la Plata, Continental Caribbean, Andean, Antilles, Chile, and the United States. A total of 182 short texts were obtained from 93 documents (of which 90 were valid, since three that appeared with different titles corresponded to the same one, so only the first one was included; and one text from a document different from the previous ones included an erratum: it actually referred to psychrometry). The first text from each valid document was selected so that the frequencies of themes came from independent texts.
Data Design and Analysis
From the methodological point of view, the subjective interpretation of meaning requires a qualitative design (Delgado, 2013; Madill & Gough, 2008; Wertz, 2014). Starting from the epistemological stance of critical realism (Delgado, 2013; Robinson, 2022), structured tabular thematic analysis (ST-TA), suitable for working with short texts, was used (Robinson, 2022). ST-TA is a version of reflexive thematic analysis influenced by well-known approaches (Boyatzis, 1998; Braun & Clarke, 2022) that also promotes simple calculations such as the frequency of themes or the percentage of agreement between researchers. It is important to note that the latter is not understood as an indicator of interrater agreement, a semantically loaded term since we are not talking about judging or scoring themes, nor about using rating scales (Robinson, 2022). If a percentage of agreement of 80% is not reached, it will be necessary to work together until consensus is reached on the issues.
Results and Discussion
After repeated reading of the 90 short texts, the first author carried out an inductive coding resulting in four themes which, in this case, constitute a system of exhaustive categories (all texts were coded) that were mutually exclusive (each text obtained a single code). This allows the simple calculations required to determine the agreement between researchers and the frequency of the themes found:
General reference to a methodological approach.
Reference to psychological or educational assessment that is carried out through the use of a type of technique.
Reference to a method of analyzing the quality of techniques.
Reference to parapsychology.
The term technique in items (b) and (c) refers to tests or instruments that were psychological in content in almost all cases. Using these themes, the second author, after several readings, deductively coded the 90 texts independently. The ST-TA method allows for the addition of new themes if the second coder considered it necessary, which did not happen.
The initial agreement, after blind coding, was 81%, so we considered that the themes adequately reflected the usual meanings reflected in the texts. Disagreements were resolved by consensus. Initially, a code called others was used, in which finally only one text was found, the one related to parapsychology (d). An example of a disagreement resolved by consensus is the case of several texts in which a well-known psychometrist or psychometric center was mentioned, which we chose to code as referring to the methodological approach in general (a), despite the fact that this theme was only the third most frequent, preceded both by that referring to the method of analysis (c), which was the most frequent, and that relating to assessment (b).
Regarding the theme Reference to a method for analyzing the quality of techniques, the most frequently used words were psychometric properties, related to the generalization of the results to other samples, validity (mainly predictive), sensitivity to change, reliability, and factorial structure, which are common contents in the psychometric field (García-Pérez, 2002; Prieto & Delgado, 2010).
Regarding the theme Reference to psychological or educational assessment carried out through the use of a type of technique, the second in frequency, most of the texts alluded to the application of psychometric tests aimed at seeking a diagnosis, to the selection of potential candidates for a given profile, or for classification.
On the other hand, it should be noted that almost all the texts include psychological content. Table 1 shows the frequency of themes with examples of sentences extracted from the coded short texts.
Table 1. Themes, Frequencies and Examples.
| a. General reference to a methodological approach | fa =22 | Example 1. "From a methodological perspective, the main research models that have been used to study learning disorders can be classified as clinical, psychometric, experimental, and psycho-pedagogical. Follow-up studies, which may include the above, deserve separate mention. Some research has integrated different methodological approaches. "Example 2. "The book by Hernstein and Murray, paradoxically the most classical, positioned in the traditional line of psychometrics, was the one that provoked great scientific debate and constituted a true publishing milestone. It defended intelligence as a unique ability that is distributed among the population following a normal bell curve, largely hereditary." |
| b. Reference to psychological or educational assessment that is carried out through the use of a type of technique | fb =32 | Example 1. "Evidently, many other psychometric tests, questionnaires, inventories, etc., are also used to evaluate other areas or relevant variables different from the specific one of addiction, such as the area of psychopathology, social adaptation, social skills, anxiety, depression, etc."Example 2. "Psychometric tests are tests frequently used in clinical psychology for the evaluation of patients, and they also help in the identification and delimitation of the severity of symptoms, specific personality profiles, and emotional and neurocognitive processes." |
| c. Reference to a method of analyzing the quality of techniques. | fc =35 | Example 1. "An adequate neuropsychological battery should be composed of a series of tests that have good psychometric properties (validity and reliability), with standardized and normalized values that serve to identify and quantify the cognitive changes derived from altered brain functions". Example 2. "Since we are using a rating scale that is in the process of validation (see next section), another of the proposed objectives was to test the psychometric properties of the instrument with the data from the sample". |
| d. Reference to parapsychology. | fd =1 | Example. "4. RELATIONSHIPS BETWEEN DOWSING, TELEPATHY, PSYCHOMETRY, AND CLAIRVOYANCE." |
Are these findings consistent with the usage of international experts? Only in part, if we think of those interviewed by Lisa Wijsen. However, they are consistent with the meanings indicated by the RAE (Real Academia Española, n.d.) according to which psychometrics is (1) the measurement of psychological phenomena and (2) the discipline that studies it.
They are also coherent with what is stated−in relation to the division of attributions between areas−about university teaching in Spain: "the practice of assessment is separated from the construction of instruments and the study of their properties, since psychometrics is included in the area of knowledge of methodology" (Sociedad Española para el Avance de la Evaluación Psicológica [Spanish Society for the Advancement of Psychological Assessment], n.d.). Also, although it is common to find the descriptor psychometrics associated with the area of personality, assessment, and psychological treatments, the above seems to indicate that, in this case, it is being used in meaning (b) of our analysis: Reference to psychological or educational assessment that is carried out through the use of a type of technique.
