Finding and removing duplicates
Brightwell's customer list has duplicates: the same business signed up twice, with the name typed differently. The email address is the best clue, once it's cleaned up.
Find duplicates by grouping on the cleaned value and keeping groups with more than one row:
SELECT lower(trim(email)) AS email, COUNT(*) AS records
FROM customers
WHERE email IS NOT NULL
GROUP BY lower(trim(email))
HAVING COUNT(*) > 1Keep one of each with ROW_NUMBER, choosing which one wins, here the oldest account:
WITH numbered AS (
SELECT id, email,
ROW_NUMBER() OVER (
PARTITION BY lower(trim(email))
ORDER BY signup_date, id
) AS rn
FROM customers
WHERE email IS NOT NULL
)
SELECT id, email FROM numbered WHERE rn = 1rn = 1 is the record to keep; rn > 1 are the copies to merge into it.
When every column is repeated (a payment typed in twice), SELECT DISTINCT on those columns
is enough to drop the copies.
Never delete duplicates you haven't looked at. Find them, check a few by eye, then decide.