| From: | Samuel Stearns <SStearns(at)internode(dot)com(dot)au> | 
|---|---|
| To: | "pgsql-performance(at)postgresql(dot)org" <pgsql-performance(at)postgresql(dot)org> | 
| Subject: | Advice on tuning slow query | 
| Date: | 2013-05-21 23:16:58 | 
| Message-ID: | CBAC86BE623FDB4E8B6225471691724291F6BB2E@EXCHMBX-ADL6-01.staff.internode.com.au | 
| Views: | Whole Thread | Raw Message | Download mbox | Resend email | 
| Thread: | |
| Lists: | pgsql-performance | 
Howdy,
Environment:
Postgres 8.4.15
Ubuntu 10.04
Syslog view def:
nms=# \d syslog
                View "public.syslog"
  Column  |            Type             | Modifiers
----------+-----------------------------+-----------
ip       | inet                        |
facility | character varying(10)       |
level    | character varying(10)       |
datetime | timestamp without time zone |
program  | character varying(25)       |
msg      | text                        |
seq      | bigint                      |
View definition:
SELECT syslog_master.ip, syslog_master.facility, syslog_master.level, syslog_master.datetime, syslog_master.program, syslog_master.msg, syslog_master.seq
   FROM syslog_master;
Rules:
syslog_insert_201304 AS
    ON INSERT TO syslog
   WHERE new.datetime >= '2013-04-01'::date AND new.datetime < '2013-05-01'::date DO INSTEAD  INSERT INTO syslog_201304 (ip, facility, level, datetime, program, msg)
  VALUES (new.ip, new.facility, new.level, new.datetime, new.program, new.msg)
syslog_insert_201305 AS
    ON INSERT TO syslog
   WHERE new.datetime >= '2013-05-01'::date AND new.datetime < '2013-06-01'::date DO INSTEAD  INSERT INTO syslog_201305 (ip, facility, level, datetime, program, msg)
  VALUES (new.ip, new.facility, new.level, new.datetime, new.program, new.msg)
syslog_insert_201306 AS
    ON INSERT TO syslog
   WHERE new.datetime >= '2013-06-01'::date AND new.datetime < '2013-07-01'::date DO INSTEAD  INSERT INTO syslog_201306 (ip, facility, level, datetime, program, msg)
  VALUES (new.ip, new.facility, new.level, new.datetime, new.program, new.msg)
syslog_insert_null AS
    ON INSERT TO syslog DO INSTEAD NOTHING
Devices table def:
nms=# \d devices
                                        Table "public.devices"
      Column      |            Type             |                      Modifiers
------------------+-----------------------------+------------------------------------------------------
id               | integer                     | not null default nextval('devices_id_seq'::regclass)
hostname         | character varying(20)       |
hostpop          | character varying(20)       |
hostgroup        | character varying(20)       |
rack             | character varying(10)       |
asset            | character varying(10)       |
ip               | inet                        |
snmprw           | character varying(20)       |
snmpro           | character varying(20)       |
snmpver          | character varying(3)        |
console          | character varying(20)       |
psu1             | character varying(20)       |
psu2             | character varying(20)       |
psu3             | character varying(20)       |
psu4             | character varying(20)       |
alias1           | character varying(20)       |
alias2           | character varying(20)       |
failure          | character varying(255)      |
modified         | timestamp without time zone | not null default now()
modified_by      | character varying(20)       |
active           | character(1)                | default 't'::bpchar
rad_secret       | character varying(20)       |
rad_atr          | character varying(40)       |
snmpdev          | integer                     |
netflow          | text                        |
cpu              | integer                     |
temp             | integer                     |
firmware_type_id | bigint                      | default 1
Indexes:
    "id_pkey" PRIMARY KEY, btree (id)
    "devices_active_index" btree (active)
    "devices_failure" btree (failure)
    "devices_hostgroup" btree (hostgroup)
    "devices_hostname" btree (hostname)
    "devices_hostpop" btree (hostpop)
    "devices_ip_index" btree (ip)
    "devices_snmprw" btree (snmprw)
Foreign-key constraints:
    "devices_firmware_type_id_fkey" FOREIGN KEY (firmware_type_id) REFERENCES firmware_type(id)
Referenced by:
    TABLE "ac_attributes" CONSTRAINT "ac_attributes_id_fkey" FOREIGN KEY (id) REFERENCES devices(id) ON DELETE CASCADE
    TABLE "acls_matrix" CONSTRAINT "acls_matrix_device_id_fkey" FOREIGN KEY (device_id) REFERENCES devices(id) ON UPDATE CASCADE ON DELETE CASCADE
    TABLE "ip_local_pool_aggregates" CONSTRAINT "ip_local_pool_aggregates_host_fkey" FOREIGN KEY (host) REFERENCES devices(id)
    TABLE "ipsla_instances" CONSTRAINT "ipsla_instances_host_fkey" FOREIGN KEY (host) REFERENCES devices(id) ON DELETE CASCADE
    TABLE "lns_attributes" CONSTRAINT "lns_attributes_id_fkey" FOREIGN KEY (id) REFERENCES devices(id) ON DELETE CASCADE
Mongroups table def:
nms=# \d mongroups
            Table "public.mongroups"
   Column   |         Type          | Modifiers
------------+-----------------------+-----------
hostgroup  | character varying(20) |
locale     | text                  |
department | character varying(20) |
Indexes:
    "ukey_hostgroup_department" UNIQUE, btree (hostgroup, department)
The following SELECT runs for 86 seconds on average:
SELECT syslog.ip,
       syslog.msg,
       syslog.datetime,
       devices.hostname,
       devices.hostpop
FROM syslog,
     devices
WHERE syslog.ip IN
    (SELECT ip
     FROM devices,
          mongroups
     WHERE (active = 't'
            OR active = 's')
       AND devices.hostgroup = mongroups.hostgroup
       AND devices.hostname || '.' || devices.hostpop = 'pe1.mel4'
       AND devices.id != '1291')
  AND datetime <= '2013-04-24 00:00:00'
  AND datetime >= '2013-04-21 00:00:00' AND syslog.ip = devices.ip AND ( devices.active = 't'
  OR devices.active = 's' );
Is there anything I can do to get the SELECT to run a little quicker.
Thank you,
Samuel Stearns
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Sergey Konoplev | 2013-05-21 23:33:16 | Re: Advice on tuning slow query | 
| Previous Message | Jaime Casanova | 2013-05-21 21:59:56 | Re: Very slow inner join query Unacceptable latency. |