Re: unorthodox use of PG for a customer

On Fri, Aug 24, 2018 at 1:19 PM David Gauthier <davegauthierpg(at)gmail(dot)com>
wrote:

> Hi Everyone:
>
> I'm going to throw this internal customer request out for ideas, even
> though I think it's a bit crazy. I'm on the brink of telling him it's
> impractical and/or inadvisable. But maybe someone has a solution.
>

Reading below, I think you're right about it being inadvisable. It seems to
me that your user, like most, just has "needs" and doesn't really want to
worry about "details" or much of anything else. We have programmers where I
would with this attitude: We list what we want -- you supply it and
maintain it. Because we only code (we don't even design much because we're
AGILE! ), we don't worry about those little details (availability,
reliability, security -- not our concern!). One thing missing from your
post is the OS involved. Linux? Windows? Other? Multiple different ones?
E.g. some users are Linux while others are Windows. {ouch}.

>
> He's writing a script/program that runs on a workstation and needs to
> write data to a DB. This process also sends work to a batch system on a
> server farm external to the workstation that will create multiple, parallel
> jobs/processes that also have to write to the DB as well. The workstation
> may have many of these jobs running at the same time. And there are 58
> workstation which all have/use locally mounted disks for this work.
>

First question. You say DB. Do you need an SQL based data base. Or do you
just need a shared data store which is easy to use in a script. Have you
considered any NOSQL type data stores such as CouchDB, MongoDB

Second question, which scripting language and what programming language?
Examples might be: Windows Powershell, Linux BASH, Windows BASH, Python,
Perl, surely not some AWK variant, R, Google GO, Rush, C/C++, and so on.

>
> At first blush, this is easy. Just create a DB on a server and have all
> those clients work with it. But he's also adamant about having the DB on
> the same server(s) that ran the script AND on the locally mounted disk. He
> said he doesn't want the overhead, dependencies and worries of anything
> like an external DB with a DBA, etc... . He also wants this to be fast.
>

So, if I understand, he wants a single shared data store for 58 clients. He
also wants the disk holding the data to be "locally mounted" to every
workstation. Depending on what "locally mounted" means to the user, I only
way that I know of to do something like this is to have the actual disk /
filesystem directly attached to a single server. All the workstations would
need to use a communication protocol (IP?) to communicate their I/O to the
"shared data" to this server to do the physical I/O. So we're talking some
protocol like NFS or CIFS (Windows "share"). The easiest way that I know to
do this sharing is to have a NAS box, such as NetApp, which really is a
server+disk "in a box" and which implements these protocols in an easy to
manage manner.

Or can each workstation have a separate, unrelated data store for its
processing. The thought then would be some way to have the application
write locally into its data store. Something else, perhaps "in line", would
replicate the data to a central store on a server. That server would then
distribute the changes back out to all 58 workstations and ??? servers so
that each will have, eventually, an identical copy of the current data; but
the stores might have inconsistencies until they synchronise.

>
> My first thought was SQLite. Apparently, they now have some sort of
> multiple, concurrent write ability. But there's no way those batch jobs on
> remote machines are going to be able to get at the locally mounted disk on
> the workstation. So I dismissed that idea. Then I thought about having 58
> PG installs, one per workstation, each serving all the jobs pertaining to
> that workstation. That could work. But 58 DB instances ? If he didn't
> like the ideal of one DBA, 58 can't be good. Still, the DB would be on the
> workstation which seems to be what he wants.
>
> I can't think of anything better. Does anyone have any ideas?
>

Yes, but the civil authorities would prosecute you if they found the body.

>
> Thanks in Advance !
>
>

--
Between infinite and short there is a big difference. -- G.H. Gonnet

Maranatha! <><
John McKown