Create one query out of two

Currently I run two queries back-to-back to correlate users with contacts.

UPDATE contacts SET user_id = u.id
FROM my_users u
JOIN phone_numbers pn ON u.phone_significant = pn.significant
WHERE contacts.owner_id = 7 AND contacts.user_id IS NULL AND contacts.id =
pn.ref_contact_id;

UPDATE contacts SET user_id = u.id
FROM my_users u
JOIN email_addresses em ON u.email = em.email
WHERE contacts.owner_id = 7 AND contacts.user_id IS NULL AND contacts.id =
em.ref_contact_id;

For some reason I cannot figure out how to combine these into one update
query. They are running slower than I'd like them to even though I have
indices on user_id, owner_id, email, and significant. So I'd like to try
them in a single query to see if that helps.

As always, thanks for your sage advice.