The following bug has been logged on the website:
Bug reference:      17853
Logged by:          Branko Radovanovic
Email address:      branko(dot)radovanovic(dot)zg(at)gmail(dot)com
PostgreSQL version: 13.4
Operating system:   Debian
Description:        
The following SQL:
values ('a')
order by 1 collate "C";
...returns an error: SQL Error [42804]: ERROR: collations are not supported
by type integer
In the above query, "1" is not an integer but a column reference, so it
should be treated as well-formed and work the same as with the actual column
label:
values ('a')
order by column1 collate "C";
Best regards,
Branko