From bec4e0e3ec71cb6a87550e43d01ca27ddc2ef72a Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Sat, 27 Apr 2024 22:11:36 +0200 Subject: [PATCH 1/3] Adds scenarios --- project/scenario2024.md | 113 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 project/scenario2024.md diff --git a/project/scenario2024.md b/project/scenario2024.md new file mode 100644 index 0000000..e6e1a45 --- /dev/null +++ b/project/scenario2024.md @@ -0,0 +1,113 @@ +# Last year's scenario/system description + +Due to the recent changes at twitter and Slack, and security problems with +Mastodon (e.g Links to an external site.), KTH decided to commission +a new system for KTH students and staff to communicate in various +constellations: individually, in closed groups according to topics or +roles, KTH wide, and open to the public, with configurable duration +of how long messages are stored. There is a prototype with all the +functionality except security by cryptography, which is your task. For +instance, access rights work correctly but nothing is encrypted or hashed. + +# Scenario/system description 2024, alternative 1 + +KTH wants to improve the security and usability of the access card system. The +cards should be optional, it should be possible to use a smartphone instead +(changes usability, whether the smartphone increases usability is up for +discussion). + +The access system handles both staff and students. Different individuals should +have different access rights. Access rights might be changed at any time +(including expiring at termination of studies or employment). + +Your task is to design the crypto that secures this system. + +# Scenario/system description 2024, alternative 2 + +KTH wants to improve the security and usability of the access card system. The +(passive) cards should be replaced by (active) key fobs that can be kept in the +pocket while opening a door (as is quite popular on modern cars, see [Smart +key][smartkey]). Possibly a smartphone can act as such an active key fob. + +Your task is to design the crypto that secures this system. + +[smartkey]: https://en.wikipedia.org/wiki/Smart_key + +# Scenario/system description 2024, alternative 3 + +People have trouble distinguishing between AI-generated and camera-recorded +photographic images. We want to be able to cryptographically verify the +authenticity of images captured by camera, thus ensuring that the images are +real and not AI-generated. + +Any photographic output from a camera should be signed by the camera. It should +not be possible to get a camera signature on an image or video it didn't +capture, so the image sensor outputs signed data that the camera then writes to +storage. + +Any image viewer/video player (like a web browser) should be able to add a +warning to the user (e.g. a red border) if the signature is missing or invalid +(i.e. it might be AI generated). + +Outline the needed crypto. Discuss the properties needed and any obstacles. +Remember to cover all relevant areas that we've covered in the course. + +## Things to think about + +Key revocation: adversarial environment, must revoke leaked keys. + +What to do with a camera if a key is revoked? Useless camera, recovery? + +Extracting the crypto HW from the camera to use it to sign other things. + +Use sidechannel attacks to extract the key. + +How to generate randomness, what source of entropy? If the HW can be tricked to +reuse the same randomness twice (zero entropy in the pool) for the same +message, we can extract the key. (For some schemes, such as Schnorr.) + +Image editing? Can sign the raw image and change image format to provide raw +(signed) and edits. Same for compression, if the compression runs in a secure +environment, it can sign the compressed image and still be trusted that it +verified the signature and that the results is still "the same". + +Taking a photo of an AI generated image? Can include a depth sensor and sign +that data with the image, to tie them together. + +Strong randomness when keys are generated, "non-overlapping" keys. + +Can track the camera by the signature, privacy concerns. +Can also use the signature to track illegal images (eg +child pornography) back to the camera and its owner. + +If only some cameras have this capability, we'll still get the problem that +people might believe pictures that are taken by "ordinary people" but censored +by the "elite journalists with an agenda" who can afford the expensive cameras +that can sign images. This is a social problem, not a technical one. + +## Alternative solutions + +Using MACs with the camera manufacturer. Manufacturer resigns it. For privacy. + +# Scenario/system description 2024, alternative 4 + +People have trouble distinguishing between bots and humans on platforms such as +Twitter (now that the real service is called X, we can actually use Twitter as +a figurative example service). The problem with bots is that one person may +appear as thousands of persons by having bot accounts act like other humans +liking tweets and posting new tweets propagating similar opinions. Thus someone +might believe that many more people share a certain opinion than is actually +the case. + +To approach this problem, we want to be able to cryptographically detect if two +users on Twitter are the same person or not, thus detecting networks of bots +belonging to the same person. For this reason we will force all users to +digitally sign all their tweets. A person making a bot network will have to +sign all tweets (or likes) with the same key. + +Someone reading a twitter feed can then detect if two tweets are signed by the +same key, and thus the same person. This can be used to detect bot networks. + +Outline the needed crypto. Discuss the properties needed and any obstacles. +Remember to cover all relevant areas that we've covered in the course. + From 0f32e29d3672c02fb9d9aaefc41cd3aef277ba33 Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Mon, 17 Jun 2024 15:29:12 +0200 Subject: [PATCH 2/3] Updates makefiles --- makefiles | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/makefiles b/makefiles index 01ac17e..45bd277 160000 --- a/makefiles +++ b/makefiles @@ -1 +1 @@ -Subproject commit 01ac17e46017837aad4ebfd4f6a31c76c99c57eb +Subproject commit 45bd27794ae5507cc16bbccab5a84a2920135adb From f4ea0c8e184292e4e0f0fd32ade4afeae578df7c Mon Sep 17 00:00:00 2001 From: Daniel Bosk Date: Mon, 17 Jun 2024 15:32:24 +0200 Subject: [PATCH 3/3] Adds first version of results evaluation script --- evaluation/.gitignore | 9 ++ evaluation/Makefile | 9 ++ evaluation/preamble.tex | 49 ++++++ evaluation/results.nw | 346 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 413 insertions(+) create mode 100644 evaluation/.gitignore create mode 100644 evaluation/Makefile create mode 100644 evaluation/preamble.tex create mode 100644 evaluation/results.nw diff --git a/evaluation/.gitignore b/evaluation/.gitignore new file mode 100644 index 0000000..5b214dd --- /dev/null +++ b/evaluation/.gitignore @@ -0,0 +1,9 @@ +ltxobj/ +results-INL1.csv +results-LAB1.csv +results.csv +results.pdf +results.sh +results.tex +submissions-INL1.csv +submissions-LAB1.csv diff --git a/evaluation/Makefile b/evaluation/Makefile new file mode 100644 index 0000000..6d19167 --- /dev/null +++ b/evaluation/Makefile @@ -0,0 +1,9 @@ +LATEXFLAGS+= -shell-escape + +.PHONY: all +all: results.pdf results.sh +Makefile: results.nw + ${NOTANGLE.mk} +INCLUDE_MAKEFILES=../makefiles +include ${INCLUDE_MAKEFILES}/tex.mk +include ${INCLUDE_MAKEFILES}/noweb.mk diff --git a/evaluation/preamble.tex b/evaluation/preamble.tex new file mode 100644 index 0000000..768337c --- /dev/null +++ b/evaluation/preamble.tex @@ -0,0 +1,49 @@ +\usepackage[utf8]{inputenc} +\usepackage[T1]{fontenc} +\usepackage[swedish,british]{babel} + +\usepackage{noweb} +\noweboptions{longchunks,longxref} +\usepackage{booktabs} + +\usepackage[natbib,style=alphabetic,maxbibnames=99]{biblatex} +\addbibresource{bibliography.bib} + +\usepackage[all]{foreign} +\renewcommand{\foreignfullfont}{} +\renewcommand{\foreignabbrfont}{} + +%\usepackage{newclude} +\usepackage{import} + +\usepackage[strict]{csquotes} +\usepackage[single]{acro} + +\usepackage{subcaption} + +\usepackage[noend]{algpseudocode} +\usepackage{xparse} + +\let\email\texttt + +\usepackage[outputdir=ltxobj]{minted} +\setminted{autogobble,linenos} + +\usepackage{pythontex} +\setpythontexoutputdir{.} +\setpythontexworkingdir{..} + +\usepackage{amsmath} +\usepackage{amssymb} +\usepackage{mathtools} +\usepackage{amsthm} +\usepackage{thmtools} +\usepackage[unq]{unique} +\DeclareMathOperator{\powerset}{\mathcal{P}} + +\usepackage[binary-units]{siunitx} + +\usepackage{didactic} + +\usepackage{hyperref} +\usepackage{cleveref} diff --git a/evaluation/results.nw b/evaluation/results.nw new file mode 100644 index 0000000..0dadb3c --- /dev/null +++ b/evaluation/results.nw @@ -0,0 +1,346 @@ +\documentclass[a4paper]{article} +\input{preamble.tex} + +% https://tex.stackexchange.com/a/205183/17418 +\def\shortyear#1{\expandafter\shortyearhelper#1} +\def\shortyearhelper#1#2#3#4{#3#4} + +\title{Results summary: tilkry\shortyear{\the\year}} +\author{Daniel Bosk} + +\begin{document} +\maketitle +\tableofcontents +@ + +\begin{pycode} +import subprocess + +def minted_output(command): + output = subprocess.run(command, capture_output=True) + print(r'\begin{minted}{text}') + print(output.stdout.decode('utf-8').strip()) + print(r'\end{minted}') +\end{pycode} + +\section{The stats} + +We'll give an overview of the stats here. +In the following section we see how they are computed. +In this section we'll just run the script and show the output here. + +\subsection{Results on INL1} + +The INL1 module has only mandatory assignments. +Let's start with the overall grades: +\begin{pycode} +minted_output(['bash', 'results.sh', 'grades', 'INL1']) +\end{pycode} + +We can see that there are some Fs. +Let's see which assignments generated Fs: +\begin{pycode} +minted_output(['bash', 'results.sh', 'fails', 'INL1']) +\end{pycode} +Most are due to the quiz. + +\subsection{Results on LAB1} + +Let's move on to LAB1. +This is the more complex part where there are optional assignments, and +depending on how the student does they'll get a grade in the scale between A +and F. +\begin{pycode} +minted_output(['bash', 'results.sh', 'grades', 'LAB1']) +\end{pycode} +Most students did well. +Particularly many got an A or B. + +Let's see which assignments generated Fs: +\begin{pycode} +minted_output(['bash', 'results.sh', 'fails', 'LAB1']) +\end{pycode} +Only one. +This is probably due to the students not having submitted some of the mandatory +assignments. +Therefore, let's have a look at which assignments missed the most submissions: +\begin{pycode} +minted_output(['bash', 'results.sh', 'missing', 'LAB1']) +\end{pycode} +We see that rather many of the mandatory assignments are missing submissions. + + +\section{The script, or, how the results are made} + +We'll write a short shell script that fetches the results from Canvas and +summarizes them in statistics. +This will be done in the same way as they are reported to LADOK, so they'll be +representative. +<>= +#!/bin/bash + +<> +<> + +main() { + <> +} + +# call main if not sourced +# https://stackoverflow.com/a/28776166/1305099 +(return 0 2>/dev/null) || main "$@" +@ + +We'll also make calls to that code in this documentation, to get the results +and how they were generated. +<>= +<> + +.PHONY: all +all: results.pdf results.sh + +results.pdf: results.tex results.sh +@ + +We'll also build this Makefile. +We're fortunately enough that [[make]] tries to remake any Makefile that it +loads. +<>= +Makefile: results.nw + ${NOTANGLE.mk} +@ + +We'll execute the script in the documentation using PythonTeX, so we need the +[[-shell-escape]] flag. +<>= +LATEXFLAGS+= -shell-escape +@ + +Finally, we'll use some automation to build. +<>= +INCLUDE_MAKEFILES=../makefiles +include ${INCLUDE_MAKEFILES}/tex.mk +include ${INCLUDE_MAKEFILES}/noweb.mk +@ + + +\subsection{Calling the functions} + +We'll take the function name as the first argument and simply call it. +If we don't have any arguments, we'll call the [[all]] function. +<>= +[[ "$1" = "" ]] && all || "$@" +<>= +all() { + <> +} +@ + + + +\subsection{Fetching the results} + +Let's start by defining the course and modules that we're interested in. +<>= +YEAR=$(date +%y) +COURSE=tilkry${YEAR} +@ + +We'll now defined two functions that can be used to fetch the desired results. +These will also act as a cache, so that we don't fetch results more than once. + +We have two modules: LAB1 and INL1. +LAB1 needs a particular module to compute the final grade ([[-S]] option). +So we need to treat that with a special case. +But INL1 can use the default. + +We'll use a function to fetch the results for each module. +It takes the module as an argument. +If the module is LAB1, we'll use the special case. + +We'll compute the grades using [[canvaslms results]], which is the same way as +the results are reported to LADOK. +However, this takes time, so we'll only do it if the file doesn't already +exist. +This way each function can call the same function to fetch the results, but it +will slow down only if they don't already exist. +<>= +fetch_results() { + local module=$1 + local file=results-${module}.csv + + echo "$file" + + [[ -f $file ]] && return + + case $module in + LAB1) + canvaslms results -c ${COURSE} -F '' \ + -A "^${module}" -S canvaslms.grades.tilkry${module} \ + > ${file} + ;; + *) + canvaslms results -c ${COURSE} -F '' \ + -A "^${module}" \ + > ${file} + ;; + esac +} +@ + +We'll do the same for fetching submission results for the individual +assignments (not the module as a whole). +There is no special case for this one. +<>= +fetch_submissions() { + local module=$1 + local file=submissions-${module}.csv + + echo "$file" + + [[ -f $file ]] && return + + canvaslms submissions -c ${COURSE} \ + -A "^${module}" \ + > ${file} +} +@ + + +\subsection{The stats} + +Let's look at the stats. +We'll add one function per stat we want. + +\subsection{Grades of modules} + +We'll start by looking at the grades of the modules. +We'll create a function that generates stats for a module, where the module is +passed as an argument. +<>= +grades() { + local modules="$@" + + for module in $modules; do + local results=`fetch_results $module` + echo + echo "${module}:" + cat $results | cut -f 4 | sort | uniq -c + done +} +@ + +If we consider running [[grades LAB1 INL1]], the result looks like this: +\begin{pycode} +minted_output(['bash', 'results.sh', 'grades', 'LAB1', 'INL1']) +\end{pycode} + + +\subsection{Which assignments generated Fs?}\label{GeneratedFs} + +We want to know this for each of the modules (LAB1, INL1). +We want a function that takes the module as an argument and then outputs the +distribution of Fs for the assignments. +We'll do this by filtering out the Fs and then count the occurrences of each +assignment. +<>= +fails() { + local modules="$@" + + for module in $modules; do + local submissions=`fetch_submissions $module` + echo + echo "${module} Fs:" + grep -E '\sFx?\s' $submissions | cut -f 2 | sort | uniq -c \ + | sort -n + done +} +@ + +If we consider running [[fails INL1]], the result looks like this: +\begin{pycode} +minted_output(['bash', 'results.sh', 'fails', 'INL1']) +\end{pycode} + +\subsection{Stats of LAB1 and INL1} + +We'll start by looking at the grades of LAB1, which is also the final grade. +<>= +echo +grades LAB1 +fails LAB1 +echo +grades INL1 +fails INL1 +@ + +\subsection{Which assignments were the least popular?} + +We now want to see which assignments the students did and which they did not. +In \cref{GeneratedFs} we saw which assignments generated Fs. +But that will only catch the assignments that students did and failed, not +assignments where students made no attempt. + +We'll create a function that takes the module as an argument and then outputs +the distribution of submissions for each assignment. +A submission is counted as a submission if it has a date. +<>= +submissions() { + local modules="$@" + + for module in $modules; do + <> + <> + echo + echo "${module} submissions:" + grep <> \ + | <> + <> + done +} +<>= +local submissions=`fetch_submissions $module` +<>= +local num_users=`cat $submissions | cut -f 3 | sort -u | wc -l` +<>= +-P '\d{4}-\d{2}-\d{2}' $submissions +<>= +cut -f 2 | sort | uniq -c | sort -rn +<>= +echo "Out of maximum ${num_users}." +@ + +If we consider running [[submissions INL1]], the result looks like this: +\begin{pycode} +minted_output(['bash', 'results.sh', 'submissions', 'INL1']) +\end{pycode} +But if we look at LAB1, which has optional assignments, we get this: +\begin{pycode} +minted_output(['bash', 'results.sh', 'submissions', 'LAB1']) +\end{pycode} + +We also add a function for the opposite, to count how many submissions an +assignment is missing. +<>= +missing() { + local modules="$@" + for module in $modules; do + <> + <> + echo + echo "${module} missing submissions:" + grep -v -P <> \ + | <> + <> + done +} +@ + +If we consider running [[missing LAB1]], the result looks like this: +\begin{pycode} +minted_output(['bash', 'results.sh', 'missing', 'LAB1']) +\end{pycode} +We can see that the assignment \enquote{Secure multi-party computation} is the +only (optional) assignment that no student did. + +\end{document}