Skip to content

Added single_intermediate_table option #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

t3t5u
Copy link
Contributor

@t3t5u t3t5u commented Jul 27, 2025

Purpose

There are two modes:

  • INSERT mode etc.: creates intermediate tables for each task
  • REPLACE mode etc.: creates only one intermediate table

In INSERT mode etc., which creates intermediate tables for each task,
due to Embulk's mechanism, tasks are generated for each input file.
Therefore, when there are a large number of input files, a large number of intermediate tables are created.

We want INSERT mode etc. to create only one intermediate table like REPLACE mode etc. does.

@t3t5u t3t5u requested a review from a team as a code owner July 27, 2025 22:57
@hiroyuki-sato
Copy link
Member

Hello, @t3t5u

Thank you for proposing the new PR.

Is this PR one-shot? Or do your organization want to contribute Embulk project itself?
As you know, @dmikurube stepped down as the active maintainer. There are no active maintainer now.

We still looking for long-term maintainers around the Embulk eco-system.

As far as I know, your organization already fork embulk-output-jdbc and extend some features on your branch.
For example, Added Azure Synapse Analytics to supported products.
Will your organization maintain both plugins?

Can I ask your organization policy?

@t3t5u
Copy link
Contributor Author

t3t5u commented Aug 1, 2025

@hiroyuki-sato

Okay, I'll take a little time to reply. 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants