8000 Errors handling during COPY FROM by ololobus · Pull Request #4 · ololobus/postgres · GitHub
[go: up one dir, main page]
Skip to content

Errors handling during COPY FROM #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Errors handling during COPY FROM #4

wants to merge 1 commit into from

Conversation

ololobus
Copy link
Owner
@ololobus ololobus commented Nov 30, 2017 •
edited
Loading

Rebased and squashed version of #3

Report line number on each ERROR/WARNING

Catch errors at InputFunctionCall inside NextCopyFrom

Tabs vs spaces consistency

Fixed possible Segmentation fault during malformed lines processing

Propagate catched SQL error code to warning

IGNORE_ERRORS COPY SQL option added (only new syntax supported)

Regression test updated to handle new COPY functionality

Fixed parser and rebase typo

@ololobus
Allow ignoring some errors during COPY FROM
5899272 
Report line number on each ERROR/WARNING

Catch errors at InputFunctionCall inside NextCopyFrom

Tabs vs spaces consistency

Fixed possible Segmentation fault during malformed lines processing

Propagate catched SQL error code to warning

IGNORE_ERRORS COPY SQL option added (only new syntax supported)

Regression test updated to handle new COPY functionality

Fixed parser and rebase typo
@ololobus ololobus changed the title Allow ignoring some errors during COPY FROM Errors handling during COPY FROM Nov 30, 2017
@ololobus ololobus mentioned this pull request Nov 30, 2017
Open
@ololobus ololobus self-assigned this Nov 30, 2017
ololobus pushed a commit that referenced this pull request Oct 22, 2018
@tglsfdc
Fix some corner-case issues in REFRESH MATERIALIZED VIEW CONCURRENTLY.
e17e905 
refresh_by_match_merge() has some issues in the way it builds a SQL
query to construct the "diff" table:

1. It doesn't require the selected unique index(es) to be indimmediate.
2. It doesn't pay attention to the particular equality semantics enforced
by a given index, but just assumes that they must be those of the column
datatype's default btree opclass.
3. It doesn't check that the indexes are btrees.
4. It's insufficiently careful to ensure that the parser will pick the
intended operator when parsing the query.  (This would have been a
security bug before CVE-2018-1058.)
5. It's not careful about indexes on system columns.

The way to fix #4 is to make use of the existing code in ri_triggers.c
for generating an arbitrary binary operator clause.  I chose to move
that to ruleutils.c, since that seems a more reasonable place to be
exporting such functionality from than ri_triggers.c.

While #1, #3, and #5 are just latent given existing feature restrictions,
and #2 doesn't arise in the core system for lack of alternate opclasses
with different equality behaviors, #4 seems like an issue worth
back-patching.  That's the bulk of the change anyway, so just back-patch
the whole thing to 9.4 where this code was introduced.

Discussion: https://postgr.es/m/13836.1521413227@sss.pgh.pa.us
ololobus pushed a commit that referenced this pull request Oct 22, 2018
@tglsfdc
Fix some corner-case issues in REFRESH MATERIALIZED VIEW CONCURRENTLY.
7d7b59a 
refresh_by_match_merge() has some issues in the way it builds a SQL
query to construct the "diff" table:

1. It doesn't require the selected unique index(es) to be indimmediate.
2. It doesn't pay attention to the particular equality semantics enforced
by a given index, but just assumes that they must be those of the column
datatype's default btree opclass.
3. It doesn't check that the indexes are btrees.
4. It's insufficiently careful to ensure that the parser will pick the
intended operator when parsing the query.  (This would have been a
security bug before CVE-2018-1058.)
5. It's not careful about indexes on system columns.

The way to fix #4 is to make use of the existing code in ri_triggers.c
for generating an arbitrary binary operator clause.  I chose to move
that to ruleutils.c, since that seems a more reasonable place to be
exporting such functionality from than ri_triggers.c.

While #1, #3, and #5 are just latent given existing feature restrictions,
and #2 doesn't arise in the core system for lack of alternate opclasses
with different equality behaviors, #4 seems like an issue worth
back-patching.  That's the bulk of the change anyway, so just back-patch
the whole thing to 9.4 where this code was introduced.

Discussion: https://postgr.es/m/13836.1521413227@sss.pgh.pa.us
@ololobus ololobus force-pushed the master branch 3 times, most recently from 5fcc467 to 5c4df12 Compare November 20, 2018 08:56
@ololobus ololobus force-pushed the master branch 2 times, most recently from d2d0925 to aebce07 Compare December 20, 2018 15:13
@ololobus ololobus force-pushed the master branch 10 times, most recently from 7c01f90 to 16485ab Compare February 8, 2019 15:40
@ololobus ololobus force-pushed the master branch 2 times, most recently from e8d65bf to 6556e62 Compare February 20, 2019 11:00
@ololobus ololobus force-pushed the master branch 3 times, most recently from 8542a98 to 2603ea1 Compare March 26, 2020 16:22
@ololobus ololobus force-pushed the master branch 4 times, most recently from 7c657da to 0e5f4f7 Compare April 6, 2020 11:24
@ololobus ololobus force-pushed the master branch 2 times, most recently from 28db880 to 7f8e356 Compare April 8, 2020 11:33
@ololobus ololobus force-pushed the master branch 2 times, most recently from b32538b to fb6f525 Compare June 25, 2020 16:39
@ololobus ololobus force-pushed the master branch 2 times, most recently from 5d7849e to afd25cc Compare July 23, 2020 14:51
@ololobus ololobus force-pushed the master branch 2 times, most recently from 2843bba to 9fcc5b8 Compare August 17, 2020 18:09
@ololobus ololobus force-pushed the master branch 3 times, most recently from 7fb7114 to 9dd5d33 Compare September 2, 2020 20:15
@ololobus ololobus force-pushed the master branch 2 times, most recently from 2435350 to 5a0dfc6 Compare September 23, 2020 14:34
@ololobus ololobus force-pushed the master branch 2 times, most recently from 5a5411e to 63ab42a Compare November 10, 2020 16:10
@ololobus ololobus force-pushed the master branch 2 times, most recently from 2943f01 to c87ec8b Compare November 17, 2020 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@ololobus ololobus

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@ololobus
0