DELETE with JOIN syntax

I am currently migrating from MySQL to PostgreSQL and I have found
that some queries do not work. For instance,

DELETE t1 FROM t1 LEFT JOIN t2 USING (column_id) WHERE t2.column_id IS NULL;

works in MySQL. This works as expected even though the MySQL
documentation does not mention the option of having a table between
the keywords DELETE and FROM.

I am trying to achieve the same affect for PostgreSQL so I tried

DELETE FROM t1 LEFT JOIN t2 USING (column_id) WHERE t2.column_id IS NULL;

and it did not work. Can someone explain to me exactly what is wrong
with this syntax?
Is a table expression produced by the JOIN allowed for a DELETE?
Im thinking that this would not work because the table expression is
not a real table and it would not make sense for DELETE to accept such
a parameter. How can I rewrite this query to achieve the same affect?
Thanks.