Re: [PATCH] Inbuilt URL validation for dynamic content generated by users.

>> I think this can help us a lot in manually searching for broken URLs
Correction: Instead of manually searching for broken URLs.

On Mon, Aug 14, 2023 at 9:17 PM Akshat Jaimini <destrex271(at)gmail(dot)com> wrote:

>
> While working on the PGweb Testing Harness we noticed that although our
> testing suite works on checking the entire website for broken links, we are
> not able to check the real time data that is present on the main website
> since the testing harness uses a development environment with some
> predefined fixtures which does not aptly satisfy this particular test.
>
> Therefore we have two patches to tackle the problem of Broken URLs at
> least in the dynamic content for the pgweb repository.
> 1. URL validation in URL specific fields and markup fields in the forms
> for adding Events, News, Organisation, Products and Professional
> Services.[form_validation_with_markdown_v03.patch]
>
> 2. A secondary service which would be run as a cronjob on the prod
> server to check tables that consist of specified URL fields and Markup
> content fields. (I am currently using the local values for database
> connection in the script; changes will be made according to the prod server
> if we decide to deploy it.)[url_scan_individual_tables.patch]
>
> Patches have been attached in the above mentioned order. Would love to
> hear your thoughts on this. I think this can help us a lot in manually
> searching for broken URLs on the website (at least in the dynamically
> generated content). I am also working on refining the crawling service
> already implemented in the Testing Harness which would be able to check for
> broken URLs.
>