Re: PG 9.3 materialized view VS Views, indexes, shared memory

>
> If you have that many different views I doubt you want that many indexes
> anyway.

​It's a datawarehouse, then each view is used by many user for each query.
Those views must be subset of the huge material table. All indexes are
needed
​

> ​Have you tried just hitting the base table and indexes directly, either
> through plain views or just direct SQL?

​I have tried each. The performances are worst querying on a subset (the
views) than querying on whole huge table when using the huge indexes

=> this is the solution I am implementing. (800 is not true, but in 10
years it maybe will be)
​Actually, I have added a boolean column on the huge table for each views​.
This is the way each view is a subset of huge table (Create View as Select
* FROM hugeTable WHERE columnX is true --etc 800 times). Then I create
800partials indexes on that column(create index...WHERE columnX is TRUE),
for each view.
This works great as the query planer chooses the partials indexes when
querying the little subset of the terrific table (potential 20bilion rows)

This is better than material views for some reasons :
- saves places on hard drive (columnX is boolean +same indexes - data for
MatViews)
- saves time generating materialised views

This is quite more complicated because in the project, the number of view
is increasing, and dynamic then :
- then adding new mat views is simple
- adding new views => adding new column on the huge table. It can take long
time to update boolean for each tuple. Then I need to truncate/bulk load
all data each time I add a new View. Other problem is dynamic number column
table was a bit tricky to implement in an ETL soft such Talend, but the
benefits are I hope great.

Nicolas PARIS

2015-03-06 2:40 GMT+01:00 Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>:

> On 2/20/15 12:09 PM, Nicolas Paris wrote:
>
>> Well it seems that max query size for CREATE INDEX is 8160 character in
>> my 9.3 postgresql version.
>> Then the only solution see is to add a new boolean field :
>> huge_table.view1
>> and change predicat to "WHERE view1=1 "
>> But I may have 800 views.. adding 800 new fields indexed to the huge
>> table is actually not a good idea. Too bad
>>
>> Any idea to solve that partial view limitation?
>>
>
> If you have that many different views I doubt you want that many indexes
> anyway.
>
> Have you tried just hitting the base table and indexes directly, either
> through plain views or just direct SQL?
>
> Also, how frequently does data change in the huge table? This sounds like
> a case where the visibility map could make a huge difference.
>
> By the way, if all the Mat Views are in one schema that's already in the
> search path, a very easy way to test this would be to create an equivalent
> set of regular views in a different schema (which you can probably do
> programmatically via pg_get_viewdef()) and then change the search_path to
> put the new schema before the old.
> --
> Jim Nasby, Data Architect, Blue Treble Consulting
> Data in Trouble? Get it in Treble! http://BlueTreble.com
>