What to index to speed up my UNION views?

From: Ashley Moran <work(at)ashleymoran(dot)me(dot)uk>
To: pgsql-general(at)postgresql(dot)org
Subject: What to index to speed up my UNION views?
Date: 2006-03-27 10:44:00
Message-ID: 200603271144.00774.work@ashleymoran.me.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I had a bright idea that has not worked quite as well as I thought.

We have a web sites for selling cars that we are trying to expand to vans,
bikes etc. We get a datafeed containing prices and technical data updated
nightly (for cars it's about 2.3GB, others are smaller). This comes into SQL
Server as one database per vehicle type, and we export it as CSV data, which
is COPY'd into Postgres.

The current version of the application uses Hibernate. Now, as the database
for each vehicle type has an idential schema I thought it would be possible
to use views to access the underlying data so we don't have to duplicate the
mappings for each data type. So for example, there is a table "capmod" which
stores vehicle models. Unfortunately, the primary key column for the table
is not unique across all dataset databases, so a model id used to identify a
car model in the car database may also identiffy a van model in the van
database.

So, I created a view like this:

CREATE OR REPLACE VIEW capmod AS
SELECT 'cap_car'::character varying::character varying(10) AS "vehicle_type",
car_capmod.cmod_code, ...
FROM cap_car.car_capmod
UNION
SELECT 'cap_lcv'::character varying::character varying(10) AS "vehicle_type",
lcv_capmod.cmod_code, ...
FROM cap_lcv.lcv_capmod;

I've removed all the actual data columns.

Effectively this makes the primary key for the view composite based on
vehicle_type and cmod_code.

The problem is that performance has taken a massive hit. Maybe the answer to
this is simple - I just need to make sure that the index on the underlying
car_capmod, lcv_capmod, XXX_capmod tables is hit in a query such as "SELECT *
FROM capmod where vehicle_type = 'cap_car' and cmod_code=1234". Failing that
I will have to include a vehicle_type column in each of the underlying
tables, but I want to avoid anything that complicates the import procedure
(which is already very slow).

I'd be very grateful for any advice

Cheers
Ashley Moran

Responses

Browse pgsql-general by date

  From Date Subject
Next Message SunWuKung 2006-03-27 10:45:05 Re: case insensitive match in unicode
Previous Message JP Glutting 2006-03-27 10:43:24 Error backing up database (Unicode)