You can use a unique index:
CREATE UNIQUE INDEX idx_foo_unique_upper_x_upper_y ON foo (UPPER(x), UPPER(y));