Which data does each test need?
Different tests need differently shaped data. Find the shape of yours below, then upload it, choose an example, or enter or paste it using the options above. One dataset is tested at a time.
Add a dataset
Upload one file: survey responses, expert ratings, scores from two occasions, or rater codes. Validex works out what kind of data it is and which reliability and validity tests it supports.
Not sure how your data should be laid out?
Click a row number or column letter to select it; Shift-click to extend, Ctrl-click (⌘-click) to add. Right-click, or use the ⋮ button, to insert, duplicate, move, sort, clear or delete. Drag across cells to select a range. Copy and paste work with Excel and Google Sheets.
Or try an example
2. Data specifications
3. Suitability
4. Results
How it’s judged
- α and ω: ≥ .90 excellent, .80 good, .70 acceptable, .60 questionable, below .60 poor; above .95 may mean redundant items
- Two-item scales: report the Spearman–Brown coefficient instead of α
- KR-20 is α for items scored 0/1
- Corrected item–total r ≥ .30; a negative value usually means an item needs reverse-coding
- Mean inter-item r between .15 and .50
α = k/(k−1) · (1 − Σσᵢ² ÷ σₓ²) ω = (Σλ)² ÷ [(Σλ)² + Σ(1 − λ²)] SB = 2r ÷ (1 + r) (odd vs even items)
Cronbach (1951); McDonald (1999); Kuder & Richardson (1937); Eisinga, te Grotenhuis & Pelzer (2013); George & Mallery (2003); Clark & Watson (1995).
How it’s judged
- ICC: below .50 poor, .50–.75 moderate, .75–.90 good, above .90 excellent
- ICC(A,1), absolute agreement, is the usual test–retest coefficient; ICC(C,1) ignores a uniform shift between occasions
- Pearson r shows association but not agreement; report it with an ICC
- Weighted κ for coded items with seven or fewer levels
- At least 30 people measured twice
ICC(A,1) = (MSR − MSE) ÷ [MSR + (k−1)MSE + k(MSC − MSE)/n] ICC(C,1) = (MSR − MSE) ÷ [MSR + (k−1)MSE]
Shrout & Fleiss (1979); McGraw & Wong (1996); Koo & Li (2016); Weir (2005).
How it’s judged
- Categories: Cohen’s κ (two raters) or Fleiss’ κ (three or more). Landis & Koch: .41–.60 moderate, .61–.80 substantial, above .80 almost perfect
- Ordered categories: weighted κ (two raters) or Krippendorff’s α (ordinal)
- Scores: ICC. Use ICC(A,1) if one rater’s score will be used, ICC(A,k) if scores are averaged
- Krippendorff’s α ≥ .80 reliable, .667–.80 for tentative conclusions only; it handles missing ratings
- Intra-rater reliability uses the same coefficients, with one rater’s occasions as the columns
κ = (pₒ − pₑ) ÷ (1 − pₑ) α = 1 − Dₒ ÷ Dₑ W = 12S ÷ [m²(n³ − n) − mΣT]
Cohen (1960, 1968); Fleiss (1971); Landis & Koch (1977); Krippendorff (2004, 2011); Kendall & Babington Smith (1939); Koo & Li (2016).
How it’s judged
Lynn (1986); Polit, Beck & Owen (2007); Lawshe (1975); Ayre & Scally (2014); Rovinelli & Hambleton (1977).
How it’s judged
- KMO ≥ .60 (≥ .80 meritorious); Bartlett’s test p < .05
- Loading λ ≥ .70; .40–.70 review; below .40 drop
- AVE ≥ .50
- CR between .70 and .95
- Cronbach’s α ≥ .70
- Corrected item–total r ≥ .30
AVE = Σλ² ÷ k CR = (Σλ)² ÷ [(Σλ)² + Σ(1 − λ²)]
Loadings come from a one-factor principal axis factoring of each construct’s items. Confirm them with CFA or PLS-SEM before publishing.
Kaiser (1974); Fornell & Larcker (1981); Hair, Black, Babin & Anderson (2019).
How it’s judged
- √AVE of each construct exceeds its correlation with every other construct
- HTMT below .85 (strict) or .90 (liberal)
- Each item correlates most with its own construct
Fornell–Larcker: √AVEᵢ > |rᵢⱼ| HTMTᵢⱼ = mean heterotrait r ÷ √(monotrait rᵢ · monotrait rⱼ)
Fornell & Larcker (1981); Henseler, Ringle & Sarstedt (2015).
Overall report
Every test you included, across all datasets and design reviews in this study. Copy the text summary into your methods section, or export the full report with all results from the button at the top right.
No dataset needed
Internal validity review
Can the effect be attributed to the cause you studied, rather than to something else?
How it’s judged
- Score ≥ 75% with no applicable threat left unaddressed
- Baseline difference |g| ≤ .05 equivalent; .05–.25 adjust statistically; above .25 not equivalent
- Report overall and differential attrition
Score = Σ ratings ÷ (2 × rated threats) g = (M₁ − M₂) ÷ s_pooled · J
Ratings are your judgement. Justify each one in the text.
Shadish, Cook & Campbell (2002); What Works Clearinghouse Procedures and Standards Handbook, v5.0 (2022).
Threats that rarely apply to this design start as N/A. Change them if they apply to your study.
Evidence calculators
For studies that compare groups. Leave them blank if they don’t apply.
Attrition
Baseline equivalence
No dataset needed
External validity review
Will the findings hold for other people, settings, treatments and times?
How it’s judged
- Sample does not differ from the population: p ≥ .05 and Cohen’s w below .30
- No category with a standardised residual beyond ±2
- Expected count of at least 5 in every category
- Generalisability review score ≥ 75%
χ² = Σ (O − E)² ÷ E, df = k − 1 w = √(χ² ÷ n)
Bracht & Glass (1968); Shadish, Cook & Campbell (2002); Cohen (1988).
Compare your sample with known population figures (enrolment records, census or industry data) on one characteristic.
Generalisability review
Rate how well the study supports generalising beyond its own sample and setting. Ratings are your judgement; justify them in the text.
Data guide
Which data does each test need?
Different tests need differently shaped data. Add one dataset per shape; Validex checks each one and tells you what it can support.
This tool is developed by Arnav (LinkedIn, opens in a new tab). Have an idea that would make it better? I’d be glad to hear from you.