Re: Column type modification in big tables

On Thu, Aug 8, 2024 at 1:06 AM sud <suds1434(at)gmail(dot)com> wrote:

>
>
> On Wed, Aug 7, 2024 at 5:00 PM Lok P <loknath(dot)73(at)gmail(dot)com> wrote:
>
>>
>>
>> On Wed, Aug 7, 2024 at 4:51 PM sud <suds1434(at)gmail(dot)com> wrote:
>>
>>>
>>>
>>> Others may correct but i think, If you don't have the FK defined on
>>> these columns you can do below.
>>>
>>>
>>> --Alter table add column which will be very fast within seconds as it
>>> will just add it to the data dictionary.
>>>
>>> ALTER TABLE tab1 ADD COLUMN new_column1 NUMERIC(3), new_column2
>>> varchar2(3);
>>>
>>>
>>> *-- Back populate the data partition wise and commit, if it's really
>>> needed*
>>>
>>> UPDATE tab1_part1 SET new_column1 = CAST(old_column1 AS NUMERIC(3)),
>>> new_column2 = CAST(old_column2 AS varchar2(3)) ;
>>> commit;
>>> UPDATE tab1_part2 SET new_column1 = CAST(old_column1 AS NUMERIC(3)),
>>> new_column2 = CAST(old_column2 AS varchar2(3)) ;
>>> commit;
>>> UPDATE tab1_part3 SET new_column1 = CAST(old_column1 AS NUMERIC(3)),
>>> new_column2 = CAST(old_column2 AS varchar2(3)) ;
>>> commit;
>>> .....
>>>
>>>
>>> *--Alter table drop old columns which will be very fast within seconds
>>> as it will just drop it from the data dictionary.*
>>> ALTER TABLE your_table DROP COLUMN old_column1, DROP COLUMN old_column2;
>>>
>>
>>
>>
>> Thank you so much.
>>
>> I understand this will be the fastest possible way to achieve the column
>> modification.
>>
>> But talking about the dropped column which will be sitting in the table
>> and consuming storage space, Is it fine to leave as is or auto vacuum will
>> remove the column values behind the scene and also anyway , once those
>> partitions will be purged they will be by default purged. Is this
>> understanding correct?
>>
>> And also will this have any impact on the partition maintenance which is
>> currently done by pg_partman as because the template table is now different
>> internally(not from outside though). Will it cause conflict because of
>> those dropped columns from the main table?
>>
>
> I think leaving the table as is after the dropping column will be fine for
> you because your regular partition maintenance/drop will slowly purge the
> historical partitions and eventually they will be removed. But if you
> update those new columns with the old column values, then autovacuum should
> also take care of removing the rows with older column values (which are
> dead actually) .
>
> Not sure if pg_partman will cause any issue ,as because the table now has
> the column data type/length changed. Others may confirm.
>

Thank you so much.

Can anybody suggest any other possible way here. As, we also need to have
the existing values be updated to the new column value here using update
command (even if it will update one partition at a time). And as I see we
have almost all the values in the column not null, which means it will
update almost ~5billion rows across all the partitions. So my question is ,
is there any parameter(like work_mem,maintenance_work_mem etc) which we can
set to make this update faster?
or any other way to get this column altered apart from this method?

>