Cleanup crash address formatting #1908

roseeichelmann · 2025-12-09T00:20:48Z

Associated issues

Testing

URL to test:
Local

Steps to test:

Start ur local db, apply migrations and metadata
Next you need to run these commands to trigger the function that updates address_display and then refresh the locations materialized view that also will need to update w the new address_display

UPDATE crashes
SET address_display = generate_crash_address(
    at_intrsct_fl,
    rpt_block_num,
    rpt_street_pfx,
    rpt_street_name,
    rpt_street_sfx,
    rpt_hwy_num,
    rpt_hwy_sfx,
    rpt_rdwy_sys_id,
    rpt_road_part_id,
    rpt_sec_block_num,
    rpt_sec_street_pfx,
    rpt_sec_street_name,
    rpt_sec_street_sfx,
    rpt_sec_hwy_num,
    rpt_sec_hwy_sfx,
    rpt_sec_rdwy_sys_id,
    rpt_sec_road_part_id
);

REFRESH MATERIALIZED VIEW location_crashes_view;

Go to the crashes list page map and click on the different points to the view the popups with the pretty addresses
Toggle the crashes list view, make sure you click the RESET button and then use the address search bar
Click on a crash to go to its details page. Toggle a crash from not on intersection to intersection to see the difference in how a crash is formatted
Edit the address for a crash by clicking on the address header and test trying to add duplicate suffix/prefixes, for example putting RD in the suffix and in the street name text boxes. See that on save the duplicate RDs wont show on the crash header ✨ '
Go to a crash with a highway number that has roadway system "LOCAL ROAD/STREET (STREET, ROAD, AVE., BLVD. PL. ...... etc etc" such as http://localhost:3002/editor/crashes/20967702 and make sure the roadway system label just been shortened to LOCAL ROAD, so the address should be something like N LOCAL ROAD 1
Go look at all the places the address_display is used, so on the fatalities map view, all the list pages, also on the EMS details page under "Possible CR3 people matches"
On locations details page, scroll down to the map and make sure that all the popups have the address displayed as expected, for both CR3 and NON-CR3 crashes
Test down migration

Ship list

Check migrations for any conflicts with latest migrations in main branch
Confirm Hasura role permissions for necessary access
Code reviewed
Product manager approved

netlify · 2025-12-09T00:21:03Z

✅ Deploy Preview for atd-vze-staging ready!

Name	Link
🔨 Latest commit	`38acff5`
🔍 Latest deploy log	https://app.netlify.com/projects/atd-vze-staging/deploys/693b1fd15e1a620008ea2333
😎 Deploy Preview	https://deploy-preview-1908--atd-vze-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

johnclary

Wow, these addresses look soo good! Thank you for taking care of all of this—there are so many parts of our codebase to update and refactor as a result of this mega improvement 🙌

I think my feedback is pretty minor. I did not run any of our ETLs, but i'll probably do a test run of the CRIS import and Socrata exporter just to be safe.

database/metadata/databases/default/tables/public_crashes_list_view.yaml

johnclary · 2025-12-17T18:13:56Z

etl/socrata_export/utils/queries.py

    id
-    address_primary
-    address_secondary
+    address_display


We're going to need to update this dataset once we launch:

https://data.austintexas.gov/Transportation-and-Mobility/Austin-Crash-Report-Data-Crash-Level-Records/y2wy-tgr5/about_data

I'll make a note in the launch checklist and possibly open it as a separate issue

database/metadata/databases/default/tables/public_locations_list_view.yaml

database/metadata/databases/default/tables/public_socrata_export_crashes_view.yaml

database/migrations/default/1767430748998_crash_address_display_trigger/up.sql

johnclary · 2025-12-17T19:05:47Z

database/migrations/default/1767430748998_crash_address_display_trigger/up.sql

+CREATE INDEX idx_crashes_address_display ON crashes(address_display);
+
+-- Function to format highway addresses
+CREATE OR REPLACE FUNCTION format_highway_address(


I love having these two separate functions for highways vs streets. Really easy to follow what's happening 🚀

johnclary · 2025-12-17T19:13:44Z

database/migrations/default/1767430748998_crash_address_display_trigger/up.sql

+            -- Only add if not a duplicate of previous (case-insensitive)
+            IF UPPER(current_part) != UPPER(prev_part) THEN
+                deduped_parts := array_append(deduped_parts, current_part);
+            END IF;


This works so well—nice!

johnclary · 2025-12-17T19:22:48Z

database/migrations/default/1767430748998_crash_address_display_trigger/up.sql

+
+
+-- Main function to generate complete crash address
+CREATE OR REPLACE FUNCTION generate_crash_address(


Not a strongly held opinion, but I think it'd be fine to simplify this function signature to accept a crash record, rather than itemizing every single column input.

im gonna leave it as is 😅

johnclary · 2025-12-17T19:23:53Z

editor/utils/formatters.ts

-  return `${crash.address_primary ? crash.address_primary : ""} ${
-    crash.address_secondary ? "& " + crash.address_secondary : ""
-  }`;
-};


editor/components/TableMapWrapper.tsx

chiaberry

This is such a giant quality improvement and so many cases to handle! I am still reviewing and wrapping my head around the SQL, but I came across this crash 21182993 and the address says "TODD LN & E BEN WHITE BLVD EB BLVD" and I dont know if it slipped through the formatting or the input is to blame.

editor/configs/emsMatchingPeopleColumns.ts

netlify · 2026-01-07T01:47:32Z

✅ Deploy Preview for atd-vze-staging ready!

Name	Link
🔨 Latest commit	`67a236f`
🔍 Latest deploy log	https://app.netlify.com/projects/atd-vze-staging/deploys/695fe3fa10b6180008df17fe
😎 Deploy Preview	https://deploy-preview-1908--atd-vze-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

…ove unnecessary if statement

roseeichelmann · 2026-01-07T20:35:17Z

@chiaberry unfortunately we are only really able to handle repeats that come right after each other like RD RD. so in the case you shared the input is just so messed up and its gonna have to be manually fixed

…/vision-zero into rose/fix_address_formatting

roseeichelmann · 2026-01-07T22:05:24Z

Alright i updated my migration to replace atd_txdot_locations with locations (so happy we finally got around to this!) and made sure my migration comes last. this should be good to go now!

chiaberry

Rose this is such a massive undertaking and huge quality improvement. 🥇

mddilley · 2026-01-07T23:32:30Z

database/metadata/databases/default/tables/public_crashes.yaml

+        - created_at
        - created_at


@roseeichelmann I'm still reviewing but it looks some column names were duplicated probably through the merging. I know how tedious the metadata can get!

ugh so frustrating i have no idea how that happened! thank you for catching this and your thorough review of the code 🙏

mddilley

I'll approve this after the metadata column duplicates that I noted are removed, but this is amazing! Love the steps to dedupe and also the highway vs. streets treatment. Easy to follow and looks ready for any future adjustments if CRIS changes anything up on us. 🚀

Also a big upgrade to have less address processing logic in the front end code. 🍄 🙌

mddilley · 2026-01-08T00:04:45Z

database/migrations/default/1767430748998_crash_address_display_trigger/up.sql

+-- Drop old columns
+
+ALTER TABLE crashes DROP COLUMN address_primary CASCADE;
+
+ALTER TABLE crashes DROP COLUMN address_secondary CASCADE;


i had forgotten that these weren't CRIS columns at all - goodbye old address columns! 👋

mateoclarke

Looks great Rose. Monster up migration but such a huge improvement. I bet this was tedious but it look so much better! 🚀

mddilley

🚢 ✨✨✨

johnclary

Test steps check out and I tested the CRIS import and socrata export ETLs. Everything looks great. Thanks, Rose! 🚢 🚢 🚢 🚢

Rose has addressed the issues! Thank you!

roseeichelmann added 6 commits December 8, 2025 18:17

metadata stuff

daef19f

migras

4db25ef

replace instances of former addresses w address_display

4a2aee1

update queries and types

0ee98e4

remove address formatter

fcf24c5

Remove mentions of address_primary

1313e6a

roseeichelmann added the WIP Work in progress label Dec 9, 2025

roseeichelmann and others added 12 commits December 10, 2025 10:12

update this trigger to handle the local road rwy sys

cb427c9

remove address_primary and address_secondary from metadata

e3baa23

make updates to triggers

2c80b85

remove this

c8c5fa9

add everything into one monstrosity of a migration

6ff0679

prettier linting

64cc9b4

fix migrations

6d4464c

add permissions for address_display to fatalities vieW

c732371

oops remove this typo

b4a551f

remove this

cd9e798

add a cascade

f3332ab

Merge branch 'main' into rose/fix_address_formatting

38acff5

roseeichelmann requested review from chiaberry, frankhereford, johnclary, mateoclarke, mddilley and tillyw December 11, 2025 19:57

roseeichelmann removed the WIP Work in progress label Dec 15, 2025

johnclary reviewed Dec 17, 2025

View reviewed changes

chiaberry reviewed Dec 23, 2025

View reviewed changes

editor/configs/emsMatchingPeopleColumns.ts Show resolved Hide resolved

roseeichelmann added 6 commits January 7, 2026 09:03

remove these edit/delete permissions from view (how did this happen

86c8d7c

add select permissions for address_display

d7c9ae7

remove this console.log

6c2049b

remove insert trigger in down migration

b1fd8f8

add separate update and insert trigger, fix SVRD not being added, rem…

c797968

…ove unnecessary if statement

add address_display to ems query

c6494c6

Merge branch 'main' into rose/fix_address_formatting

ae659ca

roseeichelmann requested review from chiaberry and johnclary January 7, 2026 20:55

frankhereford self-requested a review January 7, 2026 21:46

johnclary and others added 3 commits January 7, 2026 16:48

Merge branch 'main' into rose/fix_address_formatting

3e9f340

update atd_txdot_locations to locations and change order of migrations

41b84c5

Merge branch 'rose/fix_address_formatting' of github.com:cityofaustin…

8939aa4

…/vision-zero into rose/fix_address_formatting

chiaberry approved these changes Jan 7, 2026

View reviewed changes

mddilley reviewed Jan 7, 2026

View reviewed changes

mddilley reviewed Jan 8, 2026

View reviewed changes

mateoclarke approved these changes Jan 8, 2026

View reviewed changes

dedupe these columns

5605610

roseeichelmann requested a review from mddilley January 8, 2026 15:21

remvove last duplicate

161f8d9

mddilley approved these changes Jan 8, 2026

View reviewed changes

johnclary mentioned this pull request Jan 8, 2026

Address cleanup post-release tasks cityofaustin/atd-data-tech#26217

Closed

3 tasks

johnclary approved these changes Jan 8, 2026

View reviewed changes

Merge branch 'main' into rose/fix_address_formatting

67a236f

roseeichelmann merged commit 89d6db4 into main Jan 8, 2026
8 checks passed

roseeichelmann deleted the rose/fix_address_formatting branch January 8, 2026 17:57



		-- Main function to generate complete crash address
		CREATE OR REPLACE FUNCTION generate_crash_address(

Cleanup crash address formatting #1908

Cleanup crash address formatting #1908

Uh oh!

Conversation

roseeichelmann commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Associated issues

Testing

Ship list

Uh oh!

netlify bot commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for atd-vze-staging ready!

Uh oh!

johnclary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chiaberry left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

netlify bot commented Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for atd-vze-staging ready!

Uh oh!

roseeichelmann commented Jan 7, 2026

Uh oh!

roseeichelmann commented Jan 7, 2026

Uh oh!

chiaberry left a comment

Choose a reason for hiding this comment

Uh oh!

mddilley Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mddilley left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mateoclarke left a comment

Choose a reason for hiding this comment

Uh oh!

mddilley left a comment

Choose a reason for hiding this comment

Uh oh!

johnclary left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

roseeichelmann commented Dec 9, 2025 •

edited

Loading

netlify bot commented Dec 9, 2025 •

edited

Loading

netlify bot commented Jan 7, 2026 •

edited

Loading

mddilley Jan 7, 2026 •

edited

Loading