Add API support for crash victim images #1916

johnclary · 2025-12-16T20:43:31Z

Associated issues

Add API support for victim photo uploads atd-data-tech#26024

This PR adds a new route to our API to support victim photo uploads. Since we don't have a UI to test this with I went ahead and created pytest tests.

The setup and testing should be pretty straightforward, with the one gotcha that your auth token expires every 5 minutes. This was pretty annoying to dev around! I went just a little bit down the rabbit hole of configuring auth0 so that we could request JWTs programmatically, but Auth0 has some big scary warnings about how risky this is. So I opted to stick with copy-pasta JWTs. One maybe reasonable thing we could do would be to extend the TTL of our JWTs in staging—but considering that our staging data tends to be quite real and sensitive this doesn't seem like a great idea to me.

Testing

URL to test: Local

From the root of the repo, rebuild the api docker image

docker compose build

In the ./api directory, create your .env file based on the template file. All secrets are available in the DEVELOPMENT section of the 1pass item called Vision Zero (VZ) User & CR3 API Secrets. If you already had an .env file, note that some values have been added and removed (see diff).
From the root of the repo, start your local hasura setup including the CR3 api:

 ./vision-zero local-stack-up

In your ./editor directory, check your .env.local file to make sure that NEXT_PUBLIC_CR3_API_DOMAIN is set to http://localhost:8085. Then start your VZE: npm run dev
From the ./database directory, apply migrations and reload metadata

hasura migrate apply
hasura metadata apply

To run the API tests, follow the instructions in the ./api/tests/README.
Optionally, if you'd like to explore how the API works interactively you can spin up a container to manually execute some python commands.

# starts a container with everything you need to interact with the API
docker run -it --rm --network vision-zero_default cr3-user-api /bin/bash

import os
import pytest
import requests
from PIL import Image
import io

# set up
test_person_id = 102580  # must be a person record ID available in your local DB
token = "Bearer ...." # copy from VZE local env
api_url = f"http://cr3-user-api:5000/images/person/{test_person_id}"
headers = {"Authorization": token}

# create image
img = Image.new("RGB", (500, 500), color="blue")
image_file = io.BytesIO()
img.save(image_file, format="JPEG")
image_file.seek(0)
image_file.name = "test.jpg"

# prep / do request
files = {"file": image_file}
data = {"image_source": "test_source"}
res = requests.post(api_url, files=files, headers=headers, data=data)
assert res.status_code == 201

# checkpoint: you can inspect the person record in the DB and find the uploaded image in S3 at  /dev/images/person/102580/.jpg. here's some sql 👇

"""SELECT
   image_s3_object_key,
   image_source,
   image_original_filename
FROM
   people
WHERE
   image_source IS NOT NULL;
"""


# Get presigned URL to this image
res = requests.get(api_url, headers=headers)
assert res.status_code == 200
presigned_url = res.json()["url"]

# checkpoint: you can open presigned_url in your browser

# cleanup - delete image
res = requests.delete(api_url, headers=headers)
assert res.status_code == 200

# checkpoint: inspect DB and s3 and see that that image has been deleted

Test that the user management API works normally. Login to your local VZE as an admin and create a user with the email address <yourfirstname>.<yourlastname>[email protected]. Refresh the page and delete this user. Confirm the user was deleted successfully.
Test that the VZE CR3 download works normally—visit crash ID 13625639 and download the CR3.

Ship list

Check migrations for any conflicts with latest migrations in main branch
Confirm Hasura role permissions for necessary access
Code reviewed
Product manager approved

netlify · 2025-12-16T20:43:46Z

✅ Deploy Preview for atd-vze-staging canceled.

Name	Link
🔨 Latest commit	`c51ebcd`
🔍 Latest deploy log	https://app.netlify.com/projects/atd-vze-staging/deploys/695dccd0cf78e900086b2662

johnclary · 2025-12-16T21:15:17Z

api/server.py

+"""
 import datetime
 import json
-import os


mostly just shuffling around imports to be closer to PEP8

johnclary · 2025-12-16T21:16:14Z

api/server.py

+AWS_S3_BUCKET_ENV = getenv("AWS_S3_BUCKET_ENV", "")
+AWS_S3_CR3_LOCATION = f"{AWS_S3_BUCKET_ENV}/cr3s/pdfs"
+AWS_S3_PERSON_IMAGE_LOCATION = f"{AWS_S3_BUCKET_ENV}/images/person"
+AWS_S3_BUCKET = getenv("AWS_S3_BUCKET", "")


i reworked a few of these env vars and updated our 1pass entry accordingly. we're going to need to bring the ECS environments in sync with these changes

johnclary · 2025-12-16T21:18:09Z

api/server.py

                    {
                        "code": "invalid_header",
-                        "description": f"{e}: Unable to parse authentication token.",
+                        "description": f"Unable to parse authentication token.",


i ran the auth route through an LLM and it wisely suggested to not dump exception messages into the API response 🔒

johnclary · 2025-12-16T21:22:18Z

api/server.py

+    if not safe_person_id:
+        return jsonify(error="Missing or invalid person_id"), 400
+
+    if request.method == "GET":


For the GET endpoint, it seemed fine to continue the CR3 download pattern of returning a pre-signed URL from S3 (as opposed to streaming the image from this API).

In the VZE, we will need to create an image component that will:

Ask this API for an image of person ID 12345
The API returns a presigned URL to the VZE (if one exists)

The VZE passes the presigned URL into an <image> element

fwiw when I recently chatted with Claude about CloudFront@Edge serving profile pics in Moped, it recommended pre-signed urls (or streaming from the API as a next alternative) instead of CF@Edge. so, this sounds great to me, and it is nice to both images and CR3s working the same way! 🚀

johnclary · 2025-12-16T21:25:36Z

api/server.py

+        )
+        return jsonify(url=url)
+
+    elif request.method == "POST":


The POST endpoint allows the client to upsert an image for a given person ID. So long as the image passes our validation, we store it in S3 and save the object key and original uploaded filename in our DB. The filename of the file saved in S3 will follow the format <person_id>.<png | jpg>.

johnclary · 2025-12-23T17:13:42Z

Folks, I just reworked the docker compose setup a bit so that we have a separate compose file for testing. Charlie encountered that the entire local stack setup was previously broken until you created an ./api/.env.test file. That is no longer the case.

Charlie-Henry

I got all of the tests to work! A couple stumbling blocks for me to get things tested

you might need to create a blankapi/.env.test file to run ./vision-zero local-stack-up the first time
In step 6 I had to use --network atd-vz-data_default because my local repo is still in the old name

Love the pytest cases, easy to understand!

Charlie-Henry · 2025-12-23T17:37:47Z

api/utils/images.py

+
+
+def _handle_image_upload(person_id, file, s3, old_image_obj_key=None):
+    """Uploads an image to S3 after validating the image and removing EXIF data


Nice I would've completely forgot about EXIF data

api/utils/images.py

frankhereford

You got a lot work done in this PR, John! It worked great, and the local testing experience was 💯. I love the pytest pattern you're establishing, and all your code is clear and concise. There's a lot of great stuff in here.

api/requirements.txt

api/utils/images.py

frankhereford · 2025-12-29T19:59:09Z

api/server.py

So much 🧹 🧼 🧹 going on in this file -- thank you!

mddilley

Looks great! This was a really fun one to review since I haven't looked at an API in a while. All the tests passed, and I ran through the Python commands as well and saw the expected changes in S3 and my local DB. 🚢 🙌

mddilley · 2025-12-30T17:24:36Z

api/tests/README.md

awesome! i've been keeping the pytest coverage for the Moped API that we shelved in the back of my mind, and this will be a nice reference for a future version.

are you thinking about including this in the release testing if the API code changes? Just thinking about how dusty the Moped tests became - we've come a long way on maintaining this kind of thing and mostly wondering what you have in mind. What a level up! 🙌

mddilley · 2025-12-30T17:26:46Z

api/tests/test_person_images.py

+from PIL import Image
+import io
+
+TEST_PERSON_ID = 102580  # must be a person record ID available in your local DB


maybe this could be another env var? I've lost context on how likely it would be for this person to be removed from the data though...

I went back and forth on this too. This person ID should be in the DB for a long time to come, but I like your suggestion. I'll set this as a default person ID that also supports passing an optional override in the env 🙏

mddilley · 2025-12-30T17:27:53Z

api/tests/test_person_images.py

+@pytest.fixture
+def test_image_jpg():
+    """Create a 500x500 JPEG test image."""
+    img = Image.new("RGB", (500, 500), color="blue")


PIL is a TIL 😄 Sure was cool seeing that blue square come up with the test presigned url!

mddilley · 2025-12-30T17:46:30Z

api/utils/images.py

+    return img
+
+
+def strip_exif(img):


nice detail! this makes me think we should do the same in Moped. i'll test it out but I don't think that we are doing this for image uploads.

api/utils/images.py

mddilley · 2025-12-30T18:04:20Z

api/server.py



-@APP.route("/cr3/download/<crash_id>")
+@app.route("/cr3/download/<int:crash_id>")


this int converter is a TIL 🙏

mddilley · 2025-12-30T18:09:45Z

api/server.py

+    if not safe_person_id:
+        return jsonify(error="Missing or invalid person_id"), 400
+
+    if request.method == "GET":


fwiw when I recently chatted with Claude about CloudFront@Edge serving profile pics in Moped, it recommended pre-signed urls (or streaming from the API as a next alternative) instead of CF@Edge. so, this sounds great to me, and it is nice to both images and CR3s working the same way! 🚀

api/server.py

chiaberry

nice to get a refresher on the api and test via pytest. I am trying to wrap my head around what needs to be done when this is merged and I am really fuzzy on how this gets deployed.

johnclary · 2026-01-06T23:08:04Z

I am trying to wrap my head around what needs to be done when this is merged and I am really fuzzy on how this gets deployed.

@chiaberry despite being intentional about updating docs on this topic it is not well documented in the repo or gitbook. we can expect the API to redeploy as the last step in the image build action.

I will update this README to make that more clear 🙏

EDIT: And before we merge we'll need to figure out the most graceful way to handle the env var changes. This one might need some manual finessing of the ECS config.

mddilley

I reviewed the latest commits and ran the test suite which passed without issue - 14 passed in 7.83s. 😎 The API code in the last commit makes sense to me too. 🚢 🚀

mateoclarke

14 passed in 14.27s 🚢

chiaberry

tests pass, I can still download cr3s too. I didnt test creating a new user, but dont see why that would change with the latest updates

johnclary added 7 commits December 15, 2025 19:01

wip

b36ffbd

adding hasura stuff

52e07e7

add hasura op - ready to test with auth

8462b84

rename env var

7952a18

add Pillow requiremetn

d9ad426

add file metadata query

72c3660

add image processing, support for GET, we flyin

ff534ce

save obj key instead of filename

e559550

johnclary commented Dec 16, 2025

View reviewed changes

johnclary added 16 commits December 17, 2025 12:59

update env template

0d642e1

readme updates wip

764e6eb

update image dimension reqs

5fe5ab4

ignore more env files

9b8bf78

rename env example to env_template

907072e

update docker ignore

5147efe

delete extra docker compose

37897f4

add pillow and pytest to reqs

fe94dbc

add delete route, image_source, image_original_filename

5622a88

update readme

f19b17f

add api tests

43b8749

move stuff to utils

76b3eda

update tests and docker-compose for tests

9d5a5f3

update readmes

79c2924

update readme

58211b3

combined POST route that supports upserts

5a36f8d

johnclary requested review from mddilley, roseeichelmann and tillyw December 22, 2025 20:34

johnclary added 2 commits December 23, 2025 11:03

stringify a value

121eae2

move tests to separate docker compose and update readme

daa5b28

johnclary added 3 commits December 23, 2025 12:20

set network name

5dafdfa

add external network

c312abf

use the dang network

a377fa4

Charlie-Henry approved these changes Dec 23, 2025

View reviewed changes

frankhereford approved these changes Dec 29, 2025

View reviewed changes

mddilley approved these changes Dec 30, 2025

View reviewed changes

johnclary mentioned this pull request Jan 5, 2026

Add test coverage for entire API cityofaustin/atd-data-tech#26152

Open

chiaberry reviewed Jan 6, 2026

View reviewed changes

api/server.py Outdated Show resolved Hide resolved

chiaberry approved these changes Jan 6, 2026

View reviewed changes

johnclary added 6 commits January 6, 2026 21:10

Merge branch 'main' into john/26024-person-image-api

a7b8186

POST -> PUT

5f0aaf4

bump to pytest v9

26ff638

set debug mode based on bucket env

4906879

add env var support for person_id

a7539a1

delete old image after inserting new image

c51ebcd

johnclary requested review from Charlie-Henry, chiaberry, frankhereford and mddilley January 7, 2026 21:52

mddilley approved these changes Jan 7, 2026

View reviewed changes

This comment was marked as duplicate.

Sign in to view

mateoclarke approved these changes Jan 8, 2026

View reviewed changes

chiaberry approved these changes Jan 8, 2026

View reviewed changes



		def _handle_image_upload(person_id, file, s3, old_image_obj_key=None):
		"""Uploads an image to S3 after validating the image and removing EXIF data



		@APP.route("/cr3/download/<crash_id>")
		@app.route("/cr3/download/<int:crash_id>")

Add API support for crash victim images #1916

Are you sure you want to change the base?

Add API support for crash victim images #1916

Uh oh!

Conversation

johnclary commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Associated issues

Testing

Ship list

Uh oh!

netlify bot commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for atd-vze-staging canceled.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnclary Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mddilley Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnclary Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

johnclary commented Dec 23, 2025

Uh oh!

Charlie-Henry left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

frankhereford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mddilley left a comment

Choose a reason for hiding this comment

Uh oh!

mddilley Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mddilley Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mddilley Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chiaberry left a comment

Choose a reason for hiding this comment

Uh oh!

johnclary commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mddilley left a comment

johnclary commented Dec 16, 2025 •

edited

Loading

netlify bot commented Dec 16, 2025 •

edited

Loading

johnclary Dec 16, 2025 •

edited

Loading

mddilley Dec 30, 2025 •

edited

Loading

johnclary Dec 16, 2025 •

edited

Loading

mddilley Dec 30, 2025 •

edited

Loading

mddilley Dec 30, 2025 •

edited

Loading

mddilley Dec 30, 2025 •

edited

Loading

johnclary commented Jan 6, 2026 •

edited

Loading