Paper Structure#

This document provides guidance on how to structure a research paper in the computer security and privacy field. It includes suggestions for each section of a typical research paper, including (click to jump to section):

Title, abstract, introduction, background, related work, methods, results, discussion, conclusion, acknowledgments, references, and appendices

0. Title and Authors#

Title#

The title is the first sentence people read. Make a good impression:

• Readers might decide to read the rest of your paper or put it aside based on the title!
• Reviewers might pick your paper based on the information provided in the title.

The title of an academic publication presents a trade-off:

• Concise & interesting titles draw more readers.
• Accurate describing titles draw field experts.
• Title Goal: More readers that are field experts.

Title Don’ts:

• Avoid a title that leaves the readers guessing what your work is about.
• Do not make it too long.
• Try to not confuse the reader.

Title Archetypes#

For computer security publications there are some established and proven title patterns:

The Combination#

Combining both an interesting part (like a movie reference or cool sounding statement) with a more descriptive subtitle.

• Keepers of the Machines: Examining How System Administrators Manage Software Updates For Multiple Machines
• Where the Wild Warnings Are: Root Causes of Chrome HTTPS Certificate Errors

The Name#

Tool or attack name plus descriptive subtitle. Bit boring, but proven approach. Works for named stuff only, like tools or attacks.

• The CrossPath Attack: Disrupting the SDN Control Channel via Shared Links
• uXOM: Efficient eXecute-Only Memory on ARM Cortex-M
• ALOHA: Auxiliary Loss Optimization for Hypothesis Augmentation
• RVFuzzer: Finding Input Validation Bugs in Robotic Vehicles through Control-Guided Testing

The Sequel#

If you are following-up a previous paper, you could even reference this with a clever title:

• Initial: Stack Overflow Considered Harmful? The Impact of Copy & Paste on Android Application Security
• Follow-up: Stack Overflow Considered Helpful! Deep Learning Security Nudges Towards Stronger Cryptography

The Quote#

Very common for survey or interview publications. Presents motivation already in the title. Only applicable if you actually have a good quote.

• “Something isn’t secure, but I’m not sure how that translates into a problem”: Promoting autonomy by designing for understanding in Signal
• “There is nothing that I need to keep secret”: Sharing Practices and Concerns of Wearable Fitness Data

Replication#

Specific keyword for a replication paper. Likely required to be in the title by the conference. Replication papers are not accepted by all conferences (Accepted at: CHI, SOUPS, …).

• Replication: No One Can Hack My Mind Revisiting a Study on Expert and Non-Expert Security Practices and Advice

SoK#

Also specific keyword for a Systematization of Knowledge (SoK) paper. Papers that evaluate, systematize, and contextualize existing knowledge. Relevant conferences with SoK tracks: S&P, EuroS&P, SOUPS, PETS.

• SoK: Lessons Learned from Android Security Research for Appified Software Platforms
• SoK: Security Evaluation of Home-Based IoT Deployment

Johnny Papers#

Running joke title in Usable Security (by now very played out). Based on the first publication by Whitten and Tygar from 1999:

• Why Johnny Can’t Encrypt: A Usability Evaluation of PGP 5.0

Follow-ups to with the same title structure included:

• Why Johnny Still Can’t Encrypt: Evaluating the Usability of Email Encryption Software
• Why Johnny Still, Still Can’t Encrypt: Evaluating the Usability of a Modern PGP Client
• Helping Johnny 2.0 to encrypt his Facebook conversations

Authors#

Anonymizing: You have to anonymize (as in not include) any author names, affiliations, or other identifyable information before the actual camera ready version of a paper.

Example for formatting authors (IEEE template with OrcIDs):

In a file authors.tex:

\author{%
\IEEEauthorblockN{Mahzabin Tamanna}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    mtamann@ncsu.edu\\
}
\and
\IEEEauthorblockN{Yash Chandrani}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    ychandr@ncsu.edu\\
}
\and
\IEEEauthorblockN{Matthew Burrows}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    myburrow@ncsu.edu
}
\and
\IEEEauthorblockN{Brandon Wroblewski}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    bnwroble@ncsu.edu
}
\and
\IEEEauthorblockN{Laurie Williams}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    lawilli3@ncsu.edu
}
\and
\IEEEauthorblockN{Dominik Wermke}
\IEEEauthorblockA{%
    \textit{North Carolina State University}\\
    North Carolina, USA\\
    dwermke@ncsu.edu
}
}

Fix for IEEE authors drift for 2 and more rows, replace the “\and” that causes line break with “\linebreakand”.

% Better alignment without having to change author order
\makeatletter
\newcommand{\linebreakand}{%
  \end{@IEEEauthorhalign}
  \hfill\mbox{}\par
  \mbox{}\hfill\begin{@IEEEauthorhalign}
}
\makeatother

Alternative format if normal format does not fit well (discouraged by some conferences):

% for over three affiliations, or if they all won't fit within the width
% of the page, use this alternative format:
%

%% Authors
\author{
    \IEEEauthorblockN{%
        Dominik Wermke\IEEEauthorrefmark{1},
        Noah Wöhler\,\orcidlink{0000-0002-4172-9565}\IEEEauthorrefmark{1},
        Jan H.\ Klemmer\,\orcidlink{0000-0002-6994-7206}\IEEEauthorrefmark{2},
        Marcel Fourn\'e\,\orcidlink{0000-0003-4442-0085}\IEEEauthorrefmark{3},
        Yasemin Acar\,\orcidlink{0000-0001-7167-7383}\IEEEauthorrefmark{4}, and
        Sascha Fahl\,\orcidlink{0000-0002-5644-3316}\IEEEauthorrefmark{1}
    }
    \IEEEauthorblockA{\IEEEauthorrefmark{1}CISPA Helmholtz Center for Information Security, Germany, \hypersetup{hidelinks}\texttt{\{\href{mailto:dominik.wermke@cispa.de}{dominik.wermke},\href{mailto:noah.woehler@cispa.de}{noah.woehler},\href{mailto:fahl@cispa.de}{fahl}\}@cispa.de}}
    \IEEEauthorblockA{\IEEEauthorrefmark{2}Leibniz University Hannover, Germany, \hypersetup{hidelinks}\href{mailto:klemmer@sec.uni-hannover.de}{\texttt{klemmer@sec.uni-hannover.de}}}
    \IEEEauthorblockA{\IEEEauthorrefmark{3}Max Planck Institute for Security and Privacy, Germany, \hypersetup{hidelinks}\href{mailto:marcel.fourne@mpi-sp.org}{\texttt{marcel.fourne@mpi-sp.org}}}
    \IEEEauthorblockA{\IEEEauthorrefmark{4}George Washington University, United States, \hypersetup{hidelinks}\href{mailto:acar@gwu.edu}{\texttt{acar@gwu.edu}}}
}

Including the file in the main.tex:

\begin{document}

%don't want date printed
\date{}

\title{Committed to Trust:\\A Qualitative Study on Security \& Trust\\in Open Source Software Projects}

% Authors
\input{authors}

\maketitle

1. Abstract#

• Goal: The purpose of an abstract is to give a clear and brief overview of the topic of your work, the research questions you address, the methods you use, the results/findings you generate and the most important takeaways.
• Relevance: Most readers will not read beyond the abstract (or cite your paper) if they think it is not relevant to them based on the abstract. Make sure it clearly outlines the content of your paper and in 1–2 sentences summarizes everything you want people to remember (and put into a related work section when citing your work).

Abstract Structures#

There are many different structure approaches to abstract writing out there and each group has their own preferences. Here is my personal favorite:

The idea is to closely follow the paper content for each “section”. Each paragraph is supposed to have 2-3 sentences at most.

• Area: Explain and motivate the topic and problem of your work. Give minimal background information. Start / include in the first sentence the specific area, e.g., “Software Composition Analysis is an important step in the security software lifecycle.” for are paper on SCA.
• Problem / Challenge: Motivate the problem you are working on and make it clear why your research is important. Illustrate the research gap you looked at.
• Approach: Describe the research methods (including type of data analysis if applicable), how you generated results, and size of data sets.
  • You can help the reader by providing a linebreak / indent before the approach starts. This makes it easier to skim the abstract if they are looking for what you actually did.
  • I like the first sentence of the approach to be “citeable” (i.e., the sentence an overworked PhD student / LLM will pick for their related work). You can’t avoid your work being condensed down to a single sentence for this, so why not set up this sentence with the stuff you want your work to be cited by. This sentence should include your approach, focus, and target / data count, i.e., “We conducted 24 semi-structured interviews with software developers to investigate their processes and challenges around software composition analysis.”
• Results: Summarize the most important results/findings. Make clear why they are important and your discoveries are interesting and valuable for the community.
  • Can be merged with takeaways if overlap.
• Takeaway: Give your conclusions/recommendations/directions for future work. Make sure to illustrate the contributions to the field/community (1-2 sentences).
  • Just writing “we give recommendations” is somewhat of a cop-out and should be avoided, be more specific if possible.
  • If people are looking for a 2 sentence reference to your work, the second sentence will likely come from this section, e.g., “[Doe et al.] conducted 24 semi-structured interviews … . They found that developers struggle with … “. So you can help the referencer by providing one citable sentence with 1–2 main takeaways.

Writing Style#

• Keep it short: Somewhere between 1/4 and 1/3 of a column is common – this translates to about 200 words. Shorter abstracts look incomplete. Longer abstracts most often need to be polished more.
• Keep it punchy: Use short sentences. Long sentences will lose the readers’ focus. Avoid passive voice sentences!
• Create curiosity and avoid banality: State the conclusion concisely, avoid generic statements (“Security is important”, “We provide recommendations”), avoid overstatements (”… solving security in the open source ecosystem”).
• Don’t include references! The abstract should be an independent text (and is often displayed as such on websites etc.), parsers won’t respect your references or footnotes.
  • You don’t need to provide referenfes for claims in the abstract, instead, rehash and reference the claims in the introduction.
  • If you have to reference a paper (because you are extending it or similar), “reference” is by title and author, e.g., “We extend work of [Doe et al.] on software composition analysis by …”.
• Avoid special characters: Title and abstract are the two parts of a paper that are commonly parsed and displayed in search results. Special characters (unicode, greek characters, …) and LaTeX commands (math mode, texttt, …) will often break parsers and make your abstract incomrehensible in search results.

Example Abstract#

Example from our “Committed to Trust” paper:

Area#

Open Source Software plays an important role in many software ecosystems. Whether in operating systems, network stacks, or as low-level system drivers, software we encounter daily is permeated with code contributions from open source projects.

• Pitch-based intro, highlights importance and wide-spread distribution of open source
• Note the “software we encounter daily”, directly addressing experts and creating an emotional bind (doesn’t work for every topic).

Problem#

Decentralized development and open collaboration in open source projects introduce unique challenges: code submissions from unknown entities, limited personpower for commit or dependency reviews, and bringing new contributors up-to-date in projects’ best practices & processes.

• Pretty straight-forward problem statement, focusing on the challenges in open source.

Method#

In 27 in-depth, semi-structured interviews with owners, maintainers, and contributors from a diverse set of open source projects, we aim to investigate their security and trust practices. For this, we explore projects’ behind-the-scene processes, provided guidance & policies, as well as incident handling & encountered challenges.

• Approach clearly states used method, number of participants, and recruitment population.
• Then roughly outlines the specific covered areas.
• Note that the first sentence is a “citeable” sentence, i.e., with few modifications usable in a related work section as “[Wermke et al.] conducted 27 in-depth, semi-structured interviews with …”.

Results#

We find that our participants’ projects are highly diverse both in deployed security measures and trust processes, as well as their underlying motivations. Based on our findings, we discuss implications for the open source software ecosystem and how the research community can better support open source projects in trust and security considerations.

• Very rough outline of results (mostly because this was an explorative study and there were to many results to fit them nicely in the abstract)
• Some generic “based on our findings” statement (will be more specific for other research methods).

Takeaway#

Overall, we argue for supporting open source projects in ways that consider their individual strengths and limitations, especially in the case of smaller projects with low contributor numbers and limited access to resources.

• Not a common takeaway structure: an opinion-based argument / statement (but fits this paper because of the “researchers working together with the open source community towards a better ecosystem” pitch).

2. Introduction#

• Goal: Show the reader why the paper is interesting and important. Give an overview of the problem statement, methodology and summarize results. Provide a list of the paper’s contributions.
• Length: 1 - 1.5 pages

A well-written introduction of the paper is vital as readers form an opinion and have certain expectations after reading it. The introduction should provide enough information to the reader about the motivation, the argument, the paper structure, and the research being conducted. It may also already give an overview of the key findings.

Don’t:

• Have a vague, wrong, or disorganized introduction - it will leave a negative impression on readers.
• Try to convince the read that your research questions are hard or complicated!
• Start with empty phrases like “In today’s world” or similar.

Example Structure#

Structure for an introduction of a paper in a general (no specific previous work to focus on) security (vulnerabilities as motivation) area.

• Zoom-in: Describe the context, then zoom in on the specific area of the research (1-2 sentences).
• Hooks: These should make the reader care about the topic and want to read the paper. Recent events that relate to the research usually make good hooks (each 2-3 sentences).
• Motivation: Explain why doing this research is important (1-2 paragraphs).
• Problem Statement: Define the problem and how the research contributes to its solution. Clarify what is and is not within the scope of the paper (1-2 paragraphs).
• Research Questions: State your research questions. Each question can be followed by a very brief explanation (2-4 questions, 1-2 sentences each).
• Contribution: Point out what the papers contributions are. This can include high level major results or results teasers (2-3 sentences).
• (Optional) Paper Structure: Give a brief overview of the following sections (1-2 sentences).
• (Optional) Replication Package: Optionally, include the replication package reference to highlight it (1 sentence).

Problem Statement#

A problem statement is a common part of the introduction:

• A properly articulated problem statement makes reading and understanding of papers exceptionally easy.
• A properly articulated problem statement helps reviewers write a short overview of the paper. It is a red flag if it is hard to write that overview correctly because the paper needs refinement.
• Example: “In this work, we investigate privacy and security misconceptions by end users of cloud office applications in a user study including participants from both the U.S. and Germany. For this, we conducted two online surveys with 200 crowd workers from Amazon’s Mechanical Turk and ClickWorker. With a combination of qualitative and quantitative methods, we modeled the two surveys to explore the following research questions:” - Cloudy with a Chance of Misconceptions: Exploring Users’ Perceptions and Expectations of Security and Privacy in Cloud Office Suites

3. Background#

• Goal: Provide background information the reader needs to understand the context of the paper. Give a general understanding of topics as necessary, and refer to further reading.
• Length: Up to 1.5 pages

A background section should only be as long as absolutely necessary, and not all papers require one. If included, it should provide information that is required to understand the research and its context, but cannot be assumed to be common knowledge for the paper’s audience. Keep it short and simple, and attempt to make it as engaging as possible by highlighting the importance or impact of the given information.

Don’t:

• Cover areas that are not immediately relevant for your research
• Go into more detail than required for understanding your research
• Start at the very beginning - the readers of your paper will likely be knowledgable in computer science and security&privacy
• Include facts without a credible source
• State things as fact that are disputed in the research community - leave them out, or if especially impotant point out the conflict and give information on all points of view.

Structure#

Possible structure of a background section:

• Introductory sentence(s): Describe what topics the section covers (1-2 sentences)
• Brief paragraphs: Summarize background information by topic, citing sources for each fact (1 paragraph/topic)

Credible Sources#

Information that is presented as fact in the background section must be supported by citations of credible sources. Credible sources include published peer-reviewed works such as conference, journal and workshop publications, as well as books. Websites, blogs and news articles can generally also be used in a background section, but must be carefully reviewed for their credibility.

Distinction from Related Work#

While a background section will also include citations as sources for the given information, it will not go into details about these works, and cited sources must not be closely related to the research you present. Instead, it will focus on quickly and concisely relaying information and knowledge on a specific topic to educate the reader.

4. Related Work#

• Goal: Embed and contextualize your work into previously published peer-reviewed publications.
• Length: 1 - 1.5 pages

The purpose of the related work section is to embed your work into previously published peer-reviewed publications. Previous work that should be included in the related work section comprises conference, journal, and workshop publications. You should not include arxiv papers, Wikipedia articles, blog posts, or news articles. If you want to use them, put them in the introduction to motivate your work. However, they are not valid resources for the related work section. When selecting related publications, make sure to pick work that is technically related to your work, i.e., do not miss important related work and do not include unrelated work.

Tips:

1. Group related work in categories and put categories into subsections of the related work section
2. Describing a previous publication in the related work section should not take more than three sentences - less is usually better
3. Illustrate how your proposed work distinguishes from related work
4. A good related work section is between 1 and 1.5 pages. Up to 2 pages are ok. Shorter related work sections might create the impression of being incomplete; longer sections can be interpreted as an indicator for a minimal delta publication.

Don’t:

• cite in huge bulks - [1,2,3,4,5,6,7] is considered bad style, try to avoid more than like 5 cites in a single block.

Structure#

• Introduction: Give a short overview of the section (1-2 sentences).
  • e.g. “In this section we discuss related work in xx key areas: key area one is about; key area two focuses on; and…”
• Key Area Title: Choose a descriptive title (1 title/area)
• Key Area Body: Discuss all previous related publications in this key area (1-2 sentences per publication).
• Key Area Closing: Discuss how your proposed work is different from and extends previous related work and closes a research gap (3 - 5 sentences).
  • e.g. “In contrast, our work …”* or *“Rather than … we do … “
• Repeat: For all key areas.
• (Optional) Conclusion Summarize how your proposed work advances the field (3-5 sentences.)
  • e.g. “Our work is the first to connect key area one with key area two”* or *“In contrast to previous work, this work takes the … approach”

Examples:

• Section II in Comparing the Usability of Cryptographic APIs
• Section II in You Get Where You’re Looking For: The Impact of Information Sources on Code Security
• Section XI in Where the Wild Warnings Are: Root Causes of Chrome HTTPS Certificate Errors

Handling References#

Dominik: My usual approach for organizing references as a group is as follows:

1. Gathering related work in a shared document.
2. Writing related work using Bibtex in a bottom-up approach.

1. Gathering#

A well-working approach is to initially gather all related work in a shared document. Other collaborators can add to this document. As primary author, you should also keep an eye on other collaboration channels (Mattermost, Calls) and include related work mentioned there.

Related work includes scientific publications, but also websites and events (for the background) and relevant scientific methods (for the methods section). You can split the different reference types accross multiple tables or handle them in a combined table.

Especially if you are writing a systematization, you want to be systematic about how you collect references, i.e., “We collected related work matching the following query (software) supply chain AND (security OR privacy OR trust) AND (survey OR interview OR study) from Google Scholar and the ACM digital library.”

Dominik: I “pre-seed” my table with 10–20 references from Google Scholar searches and recent conferences. Then I go from the top through each row, decide to either include or scrap the reference, and add matching references from their related work to my table. Iterate until not references are left or you are happy with the number of references.

2. Writing#

Converting references in the table to an actual related work section includes two steps:

1. Adding a reference block for the bibliography.
2. Summarizing and referencing the reference in the related work section.

Dominik: My “bottom up” approach for references is to convert each table row into a 1–2 sentence summary with cited bibliography entry in the related work. This is likely way longer than an average related work section, so afterwards I reorder, summarize, and combine the different entries into an actual related work section.

Bibtex#

Cite Key: Preferably use something human-readable for the key (not the DOI) because Overleaf can autocomplete these, e.g., doe2021PigsCanFly.

Types#

Bitex has a number of different entry types. The most relevant for us are the following:

@inproceedings{CitekeyInproceedings,
	author    = "Holleis, Paul and Wagner, Matthias and Koolwaaij, Johan",
	title     = "Studying mobile context-aware social services in the wild",
	booktitle = "Proc. of the 6th Nordic Conf. on Human-Computer Interaction",
	series    = "NordiCHI",
	year      = 2010,
	pages     = "207--216",
	publisher = "ACM",
	address   = "New York, NY"
}

@article{CitekeyArticle,
	author   = "P. J. Cohen",
	title    = "The independence of the continuum hypothesis",
	journal  = "Proceedings of the National Academy of Sciences",
	year     = 1963,
	volume   = "50",
	number   = "6",
	pages    = "1143--1148",
}

@misc{CitekeyMisc,
	title        = "Pluto: The 'Other' Red Planet",
	author       = "{NASA}",
	howpublished = "\url{https://www.nasa.gov/nh/pluto-the-other-red-planet}",
	year         = 2015,
	note         = "Accessed: 2018-12-06"
}

Biblatex#

The biblatex library has some useful macros for references:

In \citeyear{Doe:2021}, \citeauthor{Doe:2021} investigated in 212 trials whether pigs can fly, finding that they can not~\cite{Doe:2021}.

turns into:

In 2021, Doe et al. investigated in 212 trials whether pigs can fly, finding that they can not [21].

5. Methods#

6. Results#

7. Discussion#

8. Conclusion#

9. Acknowledgements#

10. References#

11. Appendices#

Figures#

Tables#