arXiv & Preprints

arXiv & Preprints#

Preprints and arXiv uploads are public research artifacts. arXiv does not only publish the PDF; it also makes the submitted LaTeX source archive available for download. Treat that source package as publishable material: comments, disabled text, unused files, metadata, and build artifacts can still expose private discussion, reviewer notes, internal URLs, credentials, or participant-sensitive context.

Sanitize the source package. arXiv source uploads are visible and can be downloaded, parsed, and searched by other people. Do not assume LaTeX comments, ignored branches, or unused files are hidden just because they do not appear in the compiled PDF.

Motivation#

The Hidden Secrets in the arXiv project analyzed 2.7 million arXiv source submissions and found that hidden information is common:

  • 88% of source submissions contained hidden information.
  • 75% of submissions contained unique author-specific comments or ignored text, and at least 3.6% of comments were estimated to contain sensitive content.
  • 3,200 submissions had review-related keywords such as “reviewer,” “rebuttal,” or “not correct” in commented sections.
  • Their findings also include 265 API tokens, 171 passwords, 4 private keys, at least 699 editable Google Docs links, and 18 cases exposing live survey data.
  • Uploading a cleaned replacement version does not fully fix it because older arXiv versions and their sources remain accessible.

Before uploading a preprint or arXiv source package:

  1. Compile the paper from a clean checkout or Overleaf download.
  2. Run a LaTeX source cleaner on the source package:
    • Prefer COMSYS ALC-NG, the research artifact from the Hidden Secrets in the arXiv paper, if you want broader coverage of dangling files, metadata, comments, conditionals, and PDF comparison.
    • Use Google Research arxiv-latex-cleaner, the classic arXiv LaTeX cleaner, if that is already part of the project workflow, but be aware that it does not cover every metadata or hidden-content case.
  3. Remove or separately review anything the cleaner flags, especially unused files, auxiliary build files, comments, hidden branches, metadata, private notes, and image/PDF metadata.
  4. If the paper came from a review submission, remove author-anonymity leftovers such as anonymous macros, hidden acknowledgements, rebuttal notes, internal institution paths, or double-blind-only comments.
  5. Recompile the cleaned package and compare the resulting PDF against the intended preprint PDF.
  6. Inspect the cleaned source manually for TODOs, comments, internal links, participant information, keys, credentials, and author-anonymity artifacts.
  7. Upload only the cleaned package and the intended PDF.
  8. Before finalizing the arXiv submission, download or preview what arXiv will publish and check both the PDF and source package one last time.

For interview or user-study papers, also follow the interview submission guidance before posting a preprint if the consent or study protocol promised quote review, privacy review, or participant feedback.

Previous Replication & Artifacts Rebuttal Next