Replication & Artifacts

Replication Package#

Replication package and artifact submission is a separate process from the rebuttal, but it is often a good idea to start preparing the replication package during the rebuttal phase.

Don’t Publish PII! A replication package must not include any personal identifiable information of participants.

Remove Keys and Secrets! Before publishing a public version of the replication package, ensure that all keys, secrets, passwords, or any other sensitive information have been removed from the package.

Hosting artifacts: We consider a hosting process consisting of two phases, review and final publication, due to the different requirements on anonymity and hosting duration.

For Review#

For initial submission and the reviewing phase, it is crucial that we as authors remain anonymous. Therefore, we need to:

  • Check that no identifiable information is contained in the replication package.
  • Choose some anonymous hosting, e.g., via AWS. Do not use any of our infrastructure/domains (including *.ncsu.edu, *.s3c2.com, etc).

Artifact Review#

Note that USENIX has a separate, mandatory (as of 2025) artifact review process after the paper has been accepted (but before the camera ready version).

See: https://secartifacts.github.io/usenixsec2025/badges

Artifacts Available Badge: As per the new “Open-Science policy” of USENIX Security, all accepted papers are mandated to qualify for this badge. To earn this badge, the AEC must judge that the artifacts described in the “Open-Science section” of the paper have been made available for retrieval, permanently and publicly. The archived copy of the artifacts must be accessible via a long-term stable reference or DOI. For this purpose, we recommend Zenodo, but other valid hosting options include institutional and third-party digital repositories (e.g., FigShare, Dryad, or Software Heritage). Unlike previous iterations, software development repositories such as GitHub, GitLab, or personal web pages are not acceptable for this badge. Other than making the artifacts available, this badge does not mandate any further requirements on functionality, correctness, or documentation.

For Camera Ready#

In this stage, when a paper is accepted and ready for publication, we do not need to remain anonymous. While you can stick with the same hosting as before, there might be reasons to consider another hosting option.

Zenodo#

Preferably use EU-based service Zenodo, operated by the CERN. Zenodo allows you to upload your replication package and provides a DOI for your dataset, which is useful for citing it in the camera ready version.

Alternatives#

  • figshare (https://figshare.com/)
    • UK-based commercial (but free) online platform for hosting academic data and datasets.
    • jan: We used figshare several times and it always worked well.
  • osf (https://osf.io/)
    • Open Science Framework is a free and open source web application to support researchers in their project management, collaboration, and sharing of research outputs.
    • US-based, so maybe avoid until current political situation is resolved.
  • Blog post with some instructions
Previous Related Work Rebuttal Next