Re: Minmax indexes

On 9/27/13 1:43 PM, Greg Stark wrote:
>> Honestly, I think we actually need more obfuscation between what happens on the filesystem and the rest of postgres... we're starting to look at areas where that would help. For example, the recent idea of being able to truncate individual relation files and not being limited to only truncating the end of the relation. My concern in that case is that 1GB is a pretty arbitrary size that we happened to pick, so if we're going to go for more efficiency in storage we probably shouldn't just blindly stick with 1G (though of course initial implementation might do that to reduce complexity, but we better still consider where we're headed).
> The ultimate goal here would be to get the filesystem to issue a TRIM
> call so an SSD storage system can reuse the underlying blocks.
> Truncating 1GB files might be a convenient way to do it, especially if
> we have some new kind of vacuum full that can pack tuples within each
> 1GB file.
>
> But there may be easier ways to achieve the same thing. If we can
> notify the filesystem that we're not using some of the blocks in the
> middle of the file we might be able to just leave things where they
> are and have holes in the files. Or we might be better off not
> depending on truncate and just look for ways to mark entire 1GB files
> as "deprecated" and move tuples out of them until we can just remove
> that whole file.

Yeah, there's a ton of different things we might do. And dealing with free space is just one example... things like the VM give us the ability to detect areas of the heap that have gone "dormant"; imagine if we could seamlessly move that data to it's own storage, possibly compressing it at the same time. (Yes, I realize there's partitioning and tablespaces and compressing filesystems, but those are a lot more work and will never be as efficient as what the database itself can do).

Anyway, I think we're all on the same page. We should stop hijacking Alvaro's thread... ;)
--
Jim C. Nasby, Data Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net