How to speed up product code and subcode match

From: Andrus <kobruleht2(at)hot(dot)ee>
To: pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: How to speed up product code and subcode match
Date: 2023-05-23 06:53:02
Message-ID: 7181fbab-1b51-c005-2576-d287fbcd784c@hot.ee
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hi!

Price list of main products vordlusajuhinnak contains 3 prices for
product (column toode) and has 39433 products:

    create table vordlusajuhinnak( toode varchar(60), n2 numeric(8,2),
n3 numeric(8,2), n4 numeric(8,2) );

toode column in unique, may be primary key in table and contains upper
case letters, digits and - characters.

product table (toode) contains 733021 products:

    CREATE TABLE toode (
        grupp character(1),
        toode character(60) primary key,
        ... lot of other columns
      );

Both tables have pattern indexes to speed up queries:

    CREATE INDEX toode_toode_pattern_idx
        ON toode (toode bpchar_pattern_ops ASC NULLS LAST);
    -- This index is probably not used, should removed:
    CREATE INDEX vordlusajuhinnak_toode_pattern_idx ON
vordlusajuhinnak(toode bpchar_pattern_ops);

Product table as both main products and subproducts with sizes. Size is
separated by main product code always by / character:

    SHOE1-BLACK
    SHOE1-BLACK/38
    SHOE1-BLACK/41
    SHOE1-BLACK/42
    SHOE1-BLACK/43
    SHOE2/XXL
    SHOE2/L

Product codes contain upper characers only in this table also.

Trying to get prices for all products using

    create table peatoode as
    select toode.toode , n2, n3, n4
    from toode, vordlusajuhinnak
    where  toode.toode between vordlusajuhinnak.toode and
vordlusajuhinnak.toode||'/z'

Takes 4.65 hours. How to speed this query up?
Output from explain:

    > "Nested Loop  (cost=0.55..272273178.12 rows=3202240012 width=78)" "
    > ->  Seq Scan on vordlusajuhinnak  (cost=0.00..721.33 rows=39433
width=32)" "  ->  Index Only Scan using toode_pkey on toode
    > (cost=0.55..6092.62 rows=81207 width=60)" "        Index Cond: (toode
    > >= (vordlusajuhinnak.toode)::bpchar)" "        Filter:
((toode)::text <= ((vordlusajuhinnak.toode)::text || '/z'::text))"

Using  PostgreSQL 13.2, compiled by Visual C++ build 1900, 64-bit in
Windows server and psqlODBC driver.
Upgrading Postgres is possible, if this helps.

Tried also using like:

    WHERE toode.toode=vordlusajuhinnak.toode OR
      toode.toode LIKE vordlusajuhinnak.toode||'/%'

Posted also in

https://stackoverflow.com/questions/76311957/how-to-match-character-columns-with-separated-subcolumns

Andrus.

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Stephen Frost 2023-05-23 12:46:57 Re: Would PostgreSQL 16 native transparent data encryption support database level encryption?
Previous Message Ron 2023-05-23 02:10:48 Re: 15 pg_upgrade with -j