support placeholder params > 65535 for postgres. #1687

mrjnamei · 2020-05-18T08:45:29Z

mrjnamei commented

2020-05-18 08:45:29 +00:00

First-time contributor

for postgres driver only support placeholder arguments max value is 65535,
see here https://github.com/lib/pq/blob/master/conn.go#L1291
We can do some work in insertMulti method by steps .

this is useful when our params's length is bigger than 65535, or users will implement this method in their business code, which will be less useful.

but I only implement this for postgres. I don't know if other drivers have the same issue, so if you have good way to solve this will be much better.

Thanks.

for postgres driver only support placeholder arguments max value is 65535, see here [https://github.com/lib/pq/blob/master/conn.go#L1291](https://github.com/lib/pq/blob/master/conn.go#L1291) We can do some work in `insertMulti` method by steps . this is useful when our params's length is bigger than 65535, or users will implement this method in their business code, which will be less useful. but I only implement this for postgres. I don't know if other drivers have the same issue, so if you have good way to solve this will be much better. Thanks.

mrjnamei changed title from ~~WIP: support placeholder params > 65535 for postgres.~~ to support placeholder params > 65535 for postgres.

2020-05-18 08:45:55 +00:00

lunny commented

2020-05-18 13:57:34 +00:00

Users should do that themselves.

mrjnamei commented

2020-05-19 03:03:06 +00:00

First-time contributor

@lunny Thanks for reply, I think if we can provide a method to do this will be better.

If we want to insert 50000 records and each record have 20 fields, we will have 10,000 placeholder params, it's not working when we using insertMulti method, and in fact, when users using insertMulti method they even don't know the real sql executed, but we throw an error with the driver,
If we implement this in business code, it's coupling with databases, we need to implement this for different database, but lost the meaning of using orm.
If we can provide a method and don't do this in insertMulti method, will save much work, in my project many places will inert multi records with more that 65536 params (it's an async job)

what do you say?

@lunny Thanks for reply, I think if we can provide a method to do this will be better. 1. If we want to insert 50000 records and each record have 20 fields, we will have 10,000 placeholder params, it's not working when we using `insertMulti` method, and in fact, when users using `insertMulti` method they even don't know the real sql executed, but we throw an error with the driver, 2. If we implement this in business code, it's coupling with databases, we need to implement this for different database, but lost the meaning of using orm. 3. If we can provide a method and don't do this in `insertMulti` method, will save much work, in my project many places will inert multi records with more that 65536 params (it's an async job) what do you say?

mrjnamei commented

2020-05-19 03:05:18 +00:00

First-time contributor

@lunny And if the package don't provide the method I had to change to source code todo this, which is terrible for me. I'll lose the updating. 😢

@lunny And if the package don't provide the method I had to change to source code todo this, which is terrible for me. I'll lose the updating. :cry:

lunny added 1 commit 2021-06-06 11:52:18 +00:00

Merge branch 'master' into master

continuous-integration/drone/pr Build is failing

Details

a9077e4602

lunny commented

2021-06-09 10:41:37 +00:00