From: | Gunther <raj(at)gusw(dot)net> |
---|---|
To: | Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com> |
Cc: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Jeff Janes <jeff(dot)janes(at)gmail(dot)com>, pgsql-performance(at)lists(dot)postgresql(dot)org |
Subject: | Re: Out of Memory errors are frustrating as heck! |
Date: | 2019-04-21 23:58:49 |
Message-ID: | fa8e50ea-62b4-5376-c560-5fa20cfaa01d@gusw.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-performance |
After applying Tomas' patch 0002 as corrected, over 0001, same thing:
integrator=# set enable_nestloop to off;
SET
integrator=# explain analyze select * from reports.v_BusinessOperation;
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 16 to 32
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 32 to 64
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 64 to 128
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 128 to 256
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 256 to 512
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 512 to 1024
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 1024 to 2048
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 2048 to 4096
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 4096 to 8192
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 8192 to 16384
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 16384 to 32768
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 32768 to 65536
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 65536 to 131072
WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 131072 to 262144
ERROR: out of memory
DETAIL: Failed on request of size 8272 in memory context "HashBatchFiles".
And from the log:
2019-04-21 23:29:33.497 UTC [8890] LOG: database system was shut down at 2019-04-21 23:29:33 UTC
2019-04-21 23:29:33.507 UTC [8888] LOG: database system is ready to accept connections
Hashjoin 0x1732b88: initial nbatch = 16, nbuckets = 8192
2019-04-21 23:31:54.447 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 16 to 32
2019-04-21 23:31:54.447 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
2019-04-21 23:31:54.447 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 10016 free (6 chunks); 110528 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 258032 total in 31 blocks; 6208 free (0 chunks); 251824 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 17429152 bytes in 668 blocks; 1452392 free (220 chunks); 15976760 used
Hashjoin 0x1732b88: increasing nbatch to 32 because space = 4128933
Hashjoin 0x1732b88: freed 148 of 10584 tuples, space now 4071106
2019-04-21 23:31:54.450 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
2019-04-21 23:31:54.450 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 9760 free (7 chunks); 110784 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 524528 total in 63 blocks; 4416 free (2 chunks); 520112 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4213640 total in 128 blocks; 7936 free (0 chunks); 4205704 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 17629936 bytes in 698 blocks; 1450344 free (223 chunks); 16179592 used
2019-04-21 23:31:54.452 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 32 to 64
2019-04-21 23:31:54.452 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
2019-04-21 23:31:54.452 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 9760 free (7 chunks); 110784 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 524528 total in 63 blocks; 4416 free (2 chunks); 520112 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 17695648 bytes in 700 blocks; 1450344 free (223 chunks); 16245304 used
Hashjoin 0x1732b88: increasing nbatch to 64 because space = 4128826
Hashjoin 0x1732b88: freed 544 of 10584 tuples, space now 3916296
2019-04-21 23:31:54.456 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
2019-04-21 23:31:54.456 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 8224 free (7 chunks); 112320 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 1057520 total in 127 blocks; 832 free (4 chunks); 1056688 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4049360 total in 123 blocks; 7936 free (0 chunks); 4041424 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 17998648 bytes in 757 blocks; 1445224 free (225 chunks); 16553424 used
2019-04-21 23:31:54.459 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 64 to 128
2019-04-21 23:31:54.459 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
2019-04-21 23:31:54.459 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 8224 free (7 chunks); 112320 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 1057520 total in 127 blocks; 832 free (4 chunks); 1056688 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 647368 total in 10 blocks; 197536 free (13 chunks); 449832 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
TupleSort main: 4219912 total in 23 blocks; 246792 free (39 chunks); 3973120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 1057520 total in 127 blocks; 832 free (4 chunks); 1056688 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
TupleSort main: 41016 total in 3 blocks; 6504 free (6 chunks); 34512 used
Caller tuples: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 18228640 bytes in 764 blocks; 1445224 free (225 chunks); 16783416 used
Hashjoin 0x1732b88: increasing nbatch to 128 because space = 4128846
Hashjoin 0x1732b88: freed 10419 of 10585 tuples, space now 65570
2019-04-21 23:31:54.466 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
2019-04-21 23:31:54.466 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
TopMemoryContext: 120544 total in 7 blocks; 6176 free (8 chunks); 114368 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 6680 free (0 chunks); 1512 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
...
I notice now you have tons of these memory map dumps, tell me what you want to see and I will grep it out for you.
I guess the Hashjoin related things .... OK, last one before the out of memory now, and then I give you some grepped stuff...
...
TopMemoryContext: 4347672 total in 9 blocks; 41608 free (18 chunks); 4306064 used
HandleParallelMessages: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
TableSpace cache: 8192 total in 1 blocks; 2096 free (0 chunks); 6096 used
Type information cache: 24352 total in 2 blocks; 2624 free (0 chunks); 21728 used
pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 416 free (0 chunks); 7776 used
TopTransactionContext: 8192 total in 1 blocks; 5416 free (2 chunks); 2776 used
RowDescriptionContext: 8192 total in 1 blocks; 6896 free (0 chunks); 1296 used
MessageContext: 32768 total in 3 blocks; 13488 free (10 chunks); 19280 used
Operator class cache: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
smgr relation table: 32768 total in 3 blocks; 16832 free (8 chunks); 15936 used
TransactionAbortContext: 32768 total in 1 blocks; 32512 free (0 chunks); 256 used
Portal hash: 8192 total in 1 blocks; 560 free (0 chunks); 7632 used
TopPortalContext: 8192 total in 1 blocks; 7664 free (0 chunks); 528 used
PortalHoldContext: 24632 total in 2 blocks; 7392 free (0 chunks); 17240 used
PortalContext: 1482752 total in 184 blocks; 11216 free (8 chunks); 1471536 used:
ExecutorState: 2449896 total in 16 blocks; 1794968 free (3158 chunks); 654928 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
TupleSort main: 286912 total in 8 blocks; 246792 free (39 chunks); 40120 used
TupleSort main: 286912 total in 8 blocks; 246792 free (39 chunks); 40120 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 2468537520 total in 293910 blocks; 2669512 free (14 chunks); 2465868008 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
TupleSort main: 256272 total in 6 blocks; 36424 free (15 chunks); 219848 used
Caller tuples: 2097152 total in 9 blocks; 929696 free (17 chunks); 1167456 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
...
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
ExprContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
Relcache by OID: 16384 total in 2 blocks; 3512 free (2 chunks); 12872 used
CacheMemoryContext: 1101328 total in 14 blocks; 288672 free (1 chunks); 812656 used
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_toast_2619_index
index info: 2048 total in 2 blocks; 968 free (1 chunks); 1080 used: entity_id_fkidx
...
index info: 2048 total in 2 blocks; 680 free (1 chunks); 1368 used: pg_attribute_relid_attnum_index
index info: 2048 total in 2 blocks; 952 free (1 chunks); 1096 used: pg_class_oid_index
WAL record construction: 49768 total in 2 blocks; 6368 free (0 chunks); 43400 used
PrivateRefCount: 8192 total in 1 blocks; 2624 free (0 chunks); 5568 used
MdSmgr: 8192 total in 1 blocks; 6560 free (1 chunks); 1632 used
LOCALLOCK hash: 16384 total in 2 blocks; 4600 free (2 chunks); 11784 used
Timezones: 104120 total in 2 blocks; 2624 free (0 chunks); 101496 used
ErrorContext: 8192 total in 1 blocks; 7936 free (5 chunks); 256 used
Grand total: 2486488160 bytes in 294559 blocks; 6838088 free (3440 chunks); 2479650072 used
2019-04-21 23:40:23.118 UTC [8896] ERROR: out of memory
2019-04-21 23:40:23.118 UTC [8896] DETAIL: Failed on request of size 8272 in memory context "HashBatchFiles".
2019-04-21 23:40:23.118 UTC [8896] STATEMENT: explain analyze select * from reports.v_BusinessOperation;
2019-04-21 23:40:23.119 UTC [8896] LOG: could not open directory "base/pgsql_tmp/pgsql_tmp8896.2.sharedfileset": Cannot allocate memory
ok now here comes a summary grepped out:
grep 'Hash'
Hashjoin 0x1732b88: initial nbatch = 16, nbuckets = 8192
2019-04-21 23:31:54.447 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 16 to 32
2019-04-21 23:31:54.447 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 258032 total in 31 blocks; 6208 free (0 chunks); 251824 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 32 because space = 4128933
Hashjoin 0x1732b88: freed 148 of 10584 tuples, space now 4071106
2019-04-21 23:31:54.450 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 524528 total in 63 blocks; 4416 free (2 chunks); 520112 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4213640 total in 128 blocks; 7936 free (0 chunks); 4205704 used
2019-04-21 23:31:54.452 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 32 to 64
2019-04-21 23:31:54.452 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 524528 total in 63 blocks; 4416 free (2 chunks); 520112 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 64 because space = 4128826
Hashjoin 0x1732b88: freed 544 of 10584 tuples, space now 3916296
2019-04-21 23:31:54.456 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 1057520 total in 127 blocks; 832 free (4 chunks); 1056688 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4049360 total in 123 blocks; 7936 free (0 chunks); 4041424 used
2019-04-21 23:31:54.459 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 64 to 128
2019-04-21 23:31:54.459 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 1057520 total in 127 blocks; 832 free (4 chunks); 1056688 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 128 because space = 4128846
Hashjoin 0x1732b88: freed 10419 of 10585 tuples, space now 65570
2019-04-21 23:31:54.466 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 2148080 total in 257 blocks; 18160 free (6 chunks); 2129920 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 172352 total in 5 blocks; 7936 free (0 chunks); 164416 used
2019-04-21 23:32:07.174 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 128 to 256
2019-04-21 23:32:07.174 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 2148080 total in 257 blocks; 18160 free (6 chunks); 2129920 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4312208 total in 131 blocks; 7936 free (0 chunks); 4304272 used
Hashjoin 0x1732b88: increasing nbatch to 256 because space = 4128829
Hashjoin 0x1732b88: freed 10308 of 10734 tuples, space now 161815
2019-04-21 23:32:07.183 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 4312816 total in 514 blocks; 36552 free (8 chunks); 4276264 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 270920 total in 8 blocks; 7936 free (0 chunks); 262984 used
2019-04-21 23:32:18.865 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 256 to 512
2019-04-21 23:32:18.865 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 4312816 total in 514 blocks; 36552 free (8 chunks); 4276264 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 512 because space = 4128908
Hashjoin 0x1732b88: freed 398 of 10379 tuples, space now 3977787
2019-04-21 23:32:18.877 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8642288 total in 1027 blocks; 73376 free (10 chunks); 8568912 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4147928 total in 126 blocks; 7936 free (0 chunks); 4139992 used
2019-04-21 23:32:18.880 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 512 to 1024
2019-04-21 23:32:18.880 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8642288 total in 1027 blocks; 73376 free (10 chunks); 8568912 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 1024 because space = 4129008
Hashjoin 0x1732b88: freed 296 of 10360 tuples, space now 4013423
2019-04-21 23:32:18.903 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 17301232 total in 2052 blocks; 147064 free (12 chunks); 17154168 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4180784 total in 127 blocks; 7936 free (0 chunks); 4172848 used
2019-04-21 23:32:18.906 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 1024 to 2048
2019-04-21 23:32:18.906 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 17301232 total in 2052 blocks; 147064 free (12 chunks); 17154168 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 2048 because space = 4129133
Hashjoin 0x1732b88: freed 154 of 10354 tuples, space now 4068786
2019-04-21 23:32:18.946 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 34389856 total in 4102 blocks; 65176 free (14 chunks); 34324680 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4213640 total in 128 blocks; 7936 free (0 chunks); 4205704 used
2019-04-21 23:32:18.949 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 2048 to 4096
2019-04-21 23:32:18.949 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 34389856 total in 4102 blocks; 65176 free (14 chunks); 34324680 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 4096 because space = 4129035
Hashjoin 0x1732b88: freed 10167 of 10351 tuples, space now 72849
2019-04-21 23:32:19.032 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 68796256 total in 8199 blocks; 130672 free (14 chunks); 68665584 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 172352 total in 5 blocks; 7936 free (0 chunks); 164416 used
Hashjoin 0x1b3d4c8: initial nbatch = 1, nbuckets = 1024
Hashjoin 0x1b3bc08: initial nbatch = 1, nbuckets = 1024
Hashjoin 0x1b3b5d8: initial nbatch = 1, nbuckets = 1024
Hashjoin 0x15499a8: initial nbatch = 1, nbuckets = 1024
Hashjoin 0x1549978: initial nbatch = 1, nbuckets = 1024
Hashjoin 0x1b3cc88: initial nbatch = 1, nbuckets = 65536
Hashjoin 0x1553638: initial nbatch = 1, nbuckets = 65536
Hashjoin 0x1553638: initial nbatch = 1, nbuckets = 65536
Hashjoin 0x1553d38: initial nbatch = 16, nbuckets = 65536
Hashjoin 0x1b3ad98: initial nbatch = 16, nbuckets = 65536
Hashjoin 0x15538d8: initial nbatch = 16, nbuckets = 65536
2019-04-21 23:40:06.495 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 4096 to 8192
2019-04-21 23:40:06.495 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 68663008 total in 8183 blocks; 131440 free (46 chunks); 68531568 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 8192 because space = 4128997
Hashjoin 0x1732b88: freed 10555 of 10560 tuples, space now 1983
2019-04-21 23:40:06.680 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 137609056 total in 16392 blocks; 261704 free (14 chunks); 137347352 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 106640 total in 3 blocks; 7936 free (0 chunks); 98704 used
2019-04-21 23:40:06.883 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 8192 to 16384
2019-04-21 23:40:06.883 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 137592400 total in 16390 blocks; 261800 free (18 chunks); 137330600 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4246496 total in 129 blocks; 7936 free (0 chunks); 4238560 used
Hashjoin 0x1732b88: increasing nbatch to 16384 because space = 4128957
Hashjoin 0x1732b88: freed 10697 of 10764 tuples, space now 25956
2019-04-21 23:40:07.268 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 275234656 total in 32777 blocks; 523808 free (14 chunks); 274710848 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 106640 total in 3 blocks; 7936 free (0 chunks); 98704 used
2019-04-21 23:40:09.096 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 16384 to 32768
2019-04-21 23:40:09.096 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 275018128 total in 32751 blocks; 525056 free (66 chunks); 274493072 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 32768 because space = 4128890
Hashjoin 0x1732b88: freed 8 of 10809 tuples, space now 4125769
2019-04-21 23:40:10.050 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 550485856 total in 65546 blocks; 1048056 free (14 chunks); 549437800 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
2019-04-21 23:40:10.060 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 32768 to 65536
2019-04-21 23:40:10.060 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 550485856 total in 65546 blocks; 1048056 free (14 chunks); 549437800 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 65536 because space = 4128825
Hashjoin 0x1732b88: freed 20 of 10809 tuples, space now 4121380
2019-04-21 23:40:12.686 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 1100988256 total in 131083 blocks; 2096592 free (14 chunks); 1098891664 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4246496 total in 129 blocks; 7936 free (0 chunks); 4238560 used
2019-04-21 23:40:12.703 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 65536 to 131072
2019-04-21 23:40:12.703 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 1100988256 total in 131083 blocks; 2096592 free (14 chunks); 1098891664 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 131072 because space = 4129020
Hashjoin 0x1732b88: freed 2 of 10809 tuples, space now 4128291
2019-04-21 23:40:20.571 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats end =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 2201993056 total in 262156 blocks; 4193704 free (14 chunks); 2197799352 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
2019-04-21 23:40:20.602 UTC [8896] WARNING: ExecHashIncreaseNumBatches: increasing number of batches from 131072 to 262144
2019-04-21 23:40:20.602 UTC [8896] LOG: ExecHashIncreaseNumBatches ======= context stats start =======
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 2201993056 total in 262156 blocks; 4193704 free (14 chunks); 2197799352 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
Hashjoin 0x1732b88: increasing nbatch to 262144 because space = 4129055
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 32768 total in 3 blocks; 17304 free (9 chunks); 15464 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7320 free (0 chunks); 872 used
HashBatchContext: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7752 free (0 chunks); 440 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 8192 total in 1 blocks; 7936 free (0 chunks); 256 used
HashTableContext: 8192 total in 1 blocks; 7624 free (0 chunks); 568 used
HashBatchContext: 90288 total in 4 blocks; 16072 free (6 chunks); 74216 used
HashBatchFiles: 2468537520 total in 293910 blocks; 2669512 free (14 chunks); 2465868008 used
HashTableContext: 8192 total in 1 blocks; 7624 free (5 chunks); 568 used
HashBatchContext: 4279352 total in 130 blocks; 7936 free (0 chunks); 4271416 used
2019-04-21 23:40:23.118 UTC [8896] DETAIL: Failed on request of size 8272 in memory context "HashBatchFiles".
(END)
-Gunther
From | Date | Subject | |
---|---|---|---|
Next Message | Gunther | 2019-04-22 00:55:03 | Re: Out of Memory errors are frustrating as heck! |
Previous Message | Tomas Vondra | 2019-04-21 23:51:02 | Re: Out of Memory errors are frustrating as heck! |