Skip to content
This repository was archived by the owner on Feb 12, 2025. It is now read-only.

Conversation

@stepanstipl
Copy link

Original PR was GPII#696 and we agreed to combine the efforts via @klown's branch here and PR GPII#692 to universal

--- original PR text ---

This PR moves dataloader from original gpii-ops/gpii-dataloader repo into universal.

Main reasons behind this:

  • Get rid of the duplication of Dockerfile and necessity to maintain extra image and pipeline
  • Get rid of cloning the repo and installing npm modules when the container is executed (this would be slow, is prone to errors and would raise question of versioning)
  • Data loader will always have the new universal code (as it will be running in the universal image)
  • Data loader will get triggered every time there's a change to universal (as the universal image will get updated)

Most of the code was in universal already, and gpii-ops/gpii-dataloader contained basically only shell wrapper and the docker image. Work on new dataloader (GPII-3138 and GPII#626) requires changes to the existing dataloader wrapper and therefore presented a good opportunity for the move.

This is a continuation of gpii-ops/gpii-dataloader#6 PR.

Main changes:

  • Import deleteAndLoadSnapsets.sh and DataLoader.md
  • Add dataLoader dependencies to docker image (jq and curl)
  • Update data loader script to reflect new location in universal
  • Minor formatting changes
  • Add DB connectivity check to dataloader
  • Update vagrantCloudBasedContainers.sh to work with new dl location
  • Update DataLoader docs to reflect current state

I have tested this locally and plan to create corresponding PR and test for gpii-infra changes required.

@stepanstipl stepanstipl changed the title Dataloader import klown Dataloader import Oct 22, 2018
@klown klown merged commit 2d8abe9 into klown:GPII-3138B Oct 22, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants