Evaluating the Construct Validity of Text Embeddings with Application to Survey Questions [PDF]
Text embedding models from Natural Language Processing can map text data (e.g. words, sentences, documents) to supposedly meaningful numerical representations (a.k.a. text embeddings). While such models are increasingly applied in social science research, one important issue is often not addressed: the extent to which these embeddings are valid ...
arxiv +1 more source
Development and validation of an astronomy self-efficacy instrument for understanding and doing [PDF]
This paper presents a new astronomy self efficacy instrument, composed of two factors, one relating to learning astronomy content, which we call astronomy personal self efficacy, and the other relating to the use of astronomical instrumentation, specifically the use of remote robotic telescopes for data collection.
arxiv +1 more source
Improving the Reporting of Threats to Construct Validity [PDF]
Background: Construct validity concerns the use of indicators to measure a concept that is not directly measurable. Aim: This study intends to identify, categorize, assess and quantify discussions of threats to construct validity in empirical software engineering literature and use the findings to suggest ways to improve the reporting of construct ...
arxiv
A General Model Validation and Testing Tool [PDF]
We construct and propose the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model ...
arxiv +1 more source
ValiText -- a unified validation framework for computational text-based measures of social constructs [PDF]
Guidance on how to validate computational text-based measures of social constructs is fragmented. While researchers generally acknowledge the importance of validating text-based measures, they often lack a shared vocabulary and a unified framework to do so.
arxiv
A Unified Framework for Constructing Nonconvex Regularizations [PDF]
Over the past decades, many individual nonconvex methods have been proposed to achieve better sparse recovery performance in various scenarios. However, how to construct a valid nonconvex regularization function remains open in practice. In this paper, we fill in this gap by presenting a unified framework for constructing the nonconvex regularization ...
arxiv +1 more source
Constructing valid instrumental variables in generalized linear causal models from directed acyclic graphs [PDF]
Unlike other techniques of causality inference, the use of valid instrumental variables can deal with unobserved sources of both variable errors, variable omissions, and sampling bias, and still arrive at consistent estimates of average treatment effects. The only problem is to find the valid instruments.
arxiv
Standards of Validity and the Validity of Standards in Behavioral Software Engineering Research: The Perspective of Psychological Test Theory [PDF]
Background. There are some publications in software engineering research that aim at guiding researchers in assessing validity threats to their studies. Still, many researchers fail to address many aspects of validity that are essential to quantitative research on human factors. Goal.
arxiv +1 more source
Validation design I: construction of validation designs via kernel herding [PDF]
We construct validation designs $Z_m$ aimed at estimating the integrated squared prediction error of a given design $X_n$. Our approach is based on the minimization of a maximum mean discrepancy for a particular kernel, conditional on $X_n$, so that sequences of nested validation designs can be constructed incrementally by kernel herding.
arxiv
The $T_{4}$ and $G_{4}$ constructions of Costas arrays [PDF]
We examine two particular constructions of Costas arrays known as the Taylor variant of the Lempel construction, or the $T_{4}$ construction, and the variant of the Golomb construction, or the $G_{4}$ construction. We connect these constructions with the concept of Fibonacci primitive roots, and show that under the Extended Riemann Hypothesis the $T_{4}
arxiv