Submit a projectThis guide is a controlled check of how clearly your own paper is presented. Jev Paper Judge extracts PDF text and evaluates passages, structure and completeness. It does not establish that a scientific claim is true, verify a citation or predict acceptance.
A reproducible editing task
- Save two copies of a draft you are allowed to process. Keep A unchanged. In B, remove only the limitations paragraph. Keep the rest of the text, headings and PDF export settings identical. Do not submit a confidential manuscript.
- Confirm that both PDFs have selectable text and recognizable introduction, methods, results and conclusion headings. Image-only scans are not OCR'd.
- Follow the project README to set up a local copy and your own TypeSafe API key. Judge both PDFs with the same tool version. Run these commands from the project directory.
- Compare the extracted sections, limitations checklist and uncertain findings in the two JSON reports. Discard the comparison if unrelated text extraction changes or headings are not detected.
- Before editing, have the author or a domain reader check each useful finding against the passage and its original citations. Do not decide from the headline score alone.
pnpm judge original.pdf --json original.json
pnpm judge without-limitations.pdf --json without-limitations.jsonDecision criteria
| Check | Usable evidence | Human verification |
|---|---|---|
| Extraction | Body text and section order are readable in both versions | Missing tables, equations, headings or appendix text |
| Limitations finding | A relevant finding traces to the edited passage | Whether the finding applies to this paper |
| Uncertainty | Low-confidence findings remain separate | Read the original passage before revising |
| Truth and citations | Explicitly outside the automatic score | Check claims, numbers and cited sources separately |
Limits and data handling
This is an editorial comparison exercise, not an accuracy benchmark. PDF extraction and heading detection use heuristics. If you change the project's questions or weights, its reference distribution must also be rebuilt. Extracted paper content goes to TypeSafe for evaluation; check rights and data handling before using a coauthor's unpublished paper or sensitive material.
Pydantic AI + Jev Examples shows a different pattern: insert a typed judgment before an agent accepts input or runs a tool. It does not verify a paper's factual claims. If you connect a judgment to editing or publication, keep the proposed action separate from execution and retain human review.
Primary sources checked 2026-09-30
Jev Paper Judge project record · Pydantic AI + Jev project record