"SoK: From Silicon to Netlist and Beyond – Two Decades of Hardware Reverse Engineering Research" - Supplementary Material

This repository contains the supplementary material for the paper "SoK: From Silicon to Netlist and Beyond – Two Decades of Hardware Reverse Engineering Research."
Throughout this README, "artifact" refers to the code, images, and datasets released by the 187 surveyed publications.

Contents

This supplementary material contains:

• Literature collection and categorization — the literature corpus of the 187 surveyed publications, the database search query, codebooks, and all papers identified through the initial search and citation mining rounds (1–4), with screening decisions.
• Artifact evaluation notes and scripts — an evaluation note file for each artifact, together with bash scripts to download, install, patch, and run the evaluated tools and datasets.

Note on Redistribution

Due to publisher copyright restrictions and the artifacts' various licenses, we cannot redistribute the papers or artifacts. Instead, our scripts retrieve each artifact from its original source. This approach depends on the continued availability of third-party repositories and download URLs, which may not remain accessible indefinitely. If a source disappears, the corresponding artifact note documents its original location and the associated paper.

Quickstart

Docker is required for running scripts. If you do not have Docker, the following script installs it on Linux; for Windows, see 01_artifact_evaluation_and_scripts/DOCKER-README.md.

bash ./DOCKER-INSTALL-LINUX.sh

To try one exemplary artifact, the SpyDrNet netlist analysis tool — run the following from within 01_artifact_evaluation_and_scripts/:

docker buildx build -t artifacteval .
docker run -it --rm --shm-size=8G -v /var/run/docker.sock:/var/run/docker.sock artifacteval

# inside the container:
cd 20_Skouson_2020_S
bash ./install.sh   # confirm the [Y/n] prompt; leaves you in a shell with the tool ready
bash ./run.sh       # runs an exemplary command

20_Skouson_2020_S/20_Skouson_2020.md documents what we observed during evaluation and how we scored it. See 01_artifact_evaluation_and_scripts/DOCKER-README.md for Docker setup on Windows, GPU passthrough, persistent containers, and keeping downloaded data on the host.

---

Literature Collection and Categorization (00_literature/)

Literature Corpus

The corpus is provided in two formats:

• BibTeX bibliography: 00_literature/00_corpus.bib
• CSV file: 00_literature/00_corpus.csv

The CSV contains the following information for each publication:

• Year of publication
• DOI (if available)
• Paper title
• Authors
• Venue name (full)
• Venue abbreviation
• Assigned tags used during our literature categorization [1]
• Primary research focus used for statistics and aggregation (e.g., as used in Figure 1 of the paper)

Search Query and Codebook

The search query and the codebooks are provided in:

• 00_literature/01_search_query_and_codebook.pdf
• Database search keywords: A list of the keywords and search strings used during the systematic database search.
• Codebooks: Two codebooks documenting the categorization scheme:
  • Table 1 describes the tags assigned for the core topics discussed in the paper.
  • Table 2 describes the type of contribution, such as Case Study and Overview/Summary.

Papers identified through the systematic database search and subsequent citation mining iterations, alongside the specific decisions for inclusion in the full-text screening, are provided in:

• 00_literature/02_initial_search_and_citation_mining.xlsx

This file includes a Statistics sheet that provides a quantitative overview of the inclusion/exclusion decisions for the abstract screening phase. Additionally, five further sheets contain the following information for the initial search and citation mining rounds 1–4:

• Title
• Abstract
• Decision (Y/N)
• DOI (if available)
• Link count — how many corpus papers cited or were cited by this entry during mining.

---

Artifact Evaluation Notes and Scripts (01_artifact_evaluation_and_scripts/)

All materials from our evaluation of the artifacts identified in our corpus reside within this folder.

Results table (PDF)

We provide a compact overview of the results from our artifact evaluation in:

• 01_artifact_evaluation_and_scripts/00_artifact_evaluation_table.pdf

For each evaluated artifact, the table includes:

• First author name of the corresponding paper and a reference to the publication
• Year of publication
• Short description of the artifact
• Assigned scores in the categories:
  • Availability
  • Functionality
  • Reproducibility

Per-artifact notes (Markdown)

Each evaluated artifact has a dedicated folder, named:

<row_idx>_<first_author_name>_<publication_year>[_<flags>]

• row_idx — row number in the artifact evaluation table
• first_author_name — first author of the related paper
• publication_year — publication year of the related paper

Optional flags appended to the folder name indicate what the artifact contains or requires:

• _S — scripts are provided
• _Dataset — the artifact contains only data, no software to be executed
• _DataOnRequest — must be requested from the authors
• _NotAvailable — artifact was not available
• _GPURequired — an NVIDIA GPU is required to run the tool
• _EDAToolRequired — a proprietary EDA tool and a corresponding license are required (AMD/Xilinx ISE or Vivado)

Inside each folder, a dedicated note file documents our evaluation, named:

<row_idx>_<first_author_name>_<publication_year>.md

These notes include:

• Context about the publication and the referenced artifact
• Information on where to find the artifact (e.g., repository links)
• Evaluation observations and the rationale behind the assigned scores in each category
• Practical details encountered during evaluation, including any non-trivial setup steps, debugging, or compatibility issues required to get tools working

In particular, notes documenting the concrete steps needed to run the tooling (including fixes and workarounds) may be helpful for readers attempting to reproduce or build upon the evaluated artifacts.

Scripts

We provide bash scripts to install, patch, and run tools; artifacts consisting only of data instead come with a script to download them. Artifacts that were not available or must be requested from the authors have no scripts.

• install.sh - Downloads the artifact and performs all steps required to prepare it for use.
• patch.sh - Applies any fixes or adjustments required for a successful installation (e.g., correcting hardcoded paths).
• run.sh - For most tools, we have also provided scripts that run exemplary commands.
• download.sh - Downloads artifact dataset.

Note: patch.sh is not meant to be run by users. It is invoked automatically by install.sh when needed.

These scripts run within a Docker container.
The necessary information and scripts for setting up and using the container are in the following files.

• 01_artifact_evaluation_and_scripts/DOCKER-README.md - contains information on how to set up Docker and run scripts within it for Windows and Linux.
• 01_artifact_evaluation_and_scripts/DOCKER-INSTALL-LINUX.sh - a bash script that automates the installation process of Docker for Linux.
• 01_artifact_evaluation_and_scripts/Dockerfile - the Dockerfile necessary for building the image.

---