Re: How to schedule long running SQL job

From: Ron <ronljohnsonjr(at)gmail(dot)com>
To: "pgsql-admin(at)lists(dot)postgresql(dot)org" <pgsql-admin(at)lists(dot)postgresql(dot)org>
Subject: Re: How to schedule long running SQL job
Date: 2023-07-20 13:26:39
Message-ID: 4b29b4fd-db99-4bc2-7973-0d8e744b7674@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

PgAdmin is on your laptop, no?

But, of course, you can always spin up a small AWS Linux VM to run psql,
scheduled "manual" VACUUM, ANALYZE, etc.

On 7/19/23 19:20, M Sarwar wrote:
> I thought that something needs to come from AWS side but definitely I can
> try this approach of psql and Windows Scheduler.
> Thanks,
> Sarwar
>
> ----------------------------------------------------------------------------
> *From:* Ron <ronljohnsonjr(at)gmail(dot)com>
> *Sent:* Wednesday, July 19, 2023 8:09 PM
> *To:* pgsql-admin(at)lists(dot)postgresql(dot)org <pgsql-admin(at)lists(dot)postgresql(dot)org>
> *Subject:* Re: How to schedule long running SQL job
> Install psql (don't ask me how) on your laptop, then use Windows Task
> Scheduler to run your query.
>
> On 7/19/23 18:57, M Sarwar wrote:
>> From my laptop on powershell and dos prompt, it is saying tht pg_cron is
>> not recognized.
>> Do  I need anything new installation for this?
>> Thanks,
>> Sarwar
>>
>> ----------------------------------------------------------------------------
>> *From:* John Scalia <jayknowsunix(at)gmail(dot)com> <mailto:jayknowsunix(at)gmail(dot)com>
>> *Sent:* Wednesday, July 19, 2023 7:45 PM
>> *To:* M Sarwar <sarwarmd02(at)outlook(dot)com> <mailto:sarwarmd02(at)outlook(dot)com>
>> *Cc:* pgsql-admin(at)lists(dot)postgresql(dot)org
>> <mailto:pgsql-admin(at)lists(dot)postgresql(dot)org>
>> <pgsql-admin(at)lists(dot)postgresql(dot)org> <mailto:pgsql-admin(at)lists(dot)postgresql(dot)org>
>> *Subject:* Re: How to schedule long running SQL job
>> Try pg_cron. I know later versions support it.
>>
>> Sent from my iPad
>>
>>> On Jul 19, 2023, at 7:41 PM, M Sarwar <sarwarmd02(at)outlook(dot)com>
>>> <mailto:sarwarmd02(at)outlook(dot)com> wrote:
>>>
>>> 
>>> Hi,
>>> Db: Postgres
>>> Cloud: aws / RDS
>>> Tool using so far: PgAdmin
>>>
>>> SQL got aborted after running for 1 hr 40 minutes from pgadmin.
>>> I need CSV file output.
>>> What is the recommended approach for this issue.
>>>
>>> SQL:
>>>
>>> SELECT P1.FILE_ID,
>>>
>>>                 F.FILENAME,
>>>
>>>                 P1.STAGE,
>>>
>>>                 P1.SERIAL_NUMBER,
>>>
>>>                 P1.TEST_IMAGE_SET_VALUE,
>>>
>>>                 SUBSTRING(P1.SERIAL_NUMBER, '[A-Z][0-9]+') AS
>>> MCM_ID_SERIAL_NUMBER,
>>>
>>>                 P1.RUN_ID,
>>>
>>>                 SPLIT_PART(P1.RUN_ID, ':', 1) TEST_LOT
>>>
>>> FROM BRONX.TEST_PART_DETAILS_ALL_MCM P1,
>>>
>>>                 BRONX.FILES_METADATA F
>>>
>>> WHERE F.FILE_ID = P1.FILE_ID
>>>
>>>                                 AND EXISTS         (              SELECT
>>> SUBSTRING(P2.SERIAL_NUMBER, '[A-Z][0-9]+') AS MCM_ID2 ,
>>>
>>> COUNT(DISTINCT SPLIT_PART(P2.RUN_ID, ':', 1)) AS NUM_DISTINCT_TEST_LOTS
>>>
>>> FROM BRONX.TEST_PART_DETAILS_ALL_MCM P2
>>>
>>>                WHERE SUBSTRING(P2.SERIAL_NUMBER, '[A-Z][0-9]+') =
>>> SUBSTRING(P1.SERIAL_NUMBER, '[A-Z][0-9]+')
>>>
>>> GROUP BY MCM_ID2
>>>
>>> HAVING COUNT(DISTINCT SPLIT_PART(P2.RUN_ID, ':', 1))  > 1
>>>
>>>                                                                 )
>>>
>>> order BY               MCM_ID_SERIAL_NUMBER,
>>>
>>> TEST_LOT
>>>
>>> ;
>>>
>>> I used to use crontab or OEM to schedule these types jobs in the past.
>>> Now it is on RDS/AWS.
>>>
>>>
>>> Thanks,
>>> Sarwar
>>>
>
> --
> Born in Arizona, moved to Babylonia.

--
Born in Arizona, moved to Babylonia.

In response to

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Edward J. Sabol 2023-07-20 13:29:25 Re: Upgrade from PG12 to PG
Previous Message Ilya Kosmodemiansky 2023-07-20 11:43:13 Re: Upgrade from PG12 to PG