Skip to content

Commit 3d8ca75

Browse files
authored
Fix concurrency bug(s) (#85)
1 parent 53322e4 commit 3d8ca75

File tree

11 files changed

+92
-65
lines changed

11 files changed

+92
-65
lines changed

.circleci/config.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
defaults: &defaults
22
docker:
3-
- image: 087285199408.dkr.ecr.us-east-1.amazonaws.com/circle-ci-test-image-base:go1.13
3+
- image: 087285199408.dkr.ecr.us-east-1.amazonaws.com/circle-ci-test-image-base:go1.16-go111module
44
environment:
55
TERRATEST_LOG_PARSER_VERSION: NONE
66
TERRAFORM_VERSION: NONE

README.md

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -272,8 +272,6 @@ In addition, git-xargs includes several features and flags to help you:
272272
273273
git-xargs distinguishes between work that is safe to perform in parallel, such as certain git operations, and work that must be done with consideration of resource constraints, such as issuing open pull requests to GitHub's API. Therefore, git-xargs is able to perform all concurrency-safe work as quickly as possible by leveraging goroutines, while treating the more expensive open pull request API calls separately.
274274

275-
The `--max-concurrent-repos` flag allows you to specify the upper limit of concurrent goroutines that should be allowed to carry out concurrency-safe work.
276-
277275
Pull requests are handled on a separate channel so that they can be buffered and retried in accordance with rate limiting feedback git-xargs is receiving from GitHub's API.
278276
279277
This means that git-xargs performs all the work upfront that it can as quickly as possible, and then moves on to serially process the pull request jobs that have resulted from the concurrency-safe work of cloning repositories, making file changes, checking the git worktree, etc.
@@ -416,7 +414,6 @@ echo "gruntwork-io/terragrunt gruntwork-io/terratest" | git-xargs \
416414
| `--skip-pull-requests` | If you don't want any pull requests opened, but would rather have your changes committed directly to your specified branch, pass this flag. Note that it won't work if your Github repo is configured with branch protections on the branch you're trying to commit directly to! Default: `false`. | Boolean | No |
417415
| `--skip-archived-repos` | If you want to exclude archived (read-only) repositories from the list of targeted repos, pass this flag. Default: `false`. | Boolean | No |
418416
| `--dry-run` | If you are in the process of testing out `git-xargs` or your initial set of targeted repos, but you don't want to make any changes via the Github API (pushing your local changes or opening pull requests) you can pass the dry-run flag. This is useful because the output report will still tell you which repos would have been affected, without actually making changes via the Github API to your remote repositories. Default: `false`. | Boolean | No |
419-
| `--max-concurrent-repos` | Limits the number of concurrent processed repositories. This is only useful if you encounter issues and need throttling when running on a very large number of repos. Default:`0` (Unlimited). | Integer | No |
420417
| `--draft` | Whether to open pull requests in draft mode. Draft pull requests are available for public GitHub repositories and private repositories in GitHub tiered accounts. See [Draft Pull Requests](https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests#draft-pull-requests) for more details. Default: false. | Boolean | No |
421418
| `--seconds-between-prs` | The number of seconds to wait between opening serial pull requests. If you are being rate limited, continue to increase this value until rate limiting eases. Note, this value cannot be negative, so if you pass a value less than 1, the seconds to wait between pull requests will be set to 1 second. Default: `1` second. | Integer | No |
422419
| `--max-pr-retries` | The number of seconds to wait between opening serial pull requests. If you are being rate limited, continue to increase this value until rate limiting eases. Default: `3` seconds. | Integer | No |

common/common.go

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ const (
1515
BaseBranchFlagName = "base-branch-name"
1616
PullRequestTitleFlagName = "pull-request-title"
1717
PullRequestDescriptionFlagName = "pull-request-description"
18-
MaxConcurrentReposFlagName = "max-concurrent-repos"
1918
SecondsToWaitBetweenPrsFlagName = "seconds-between-prs"
2019
DefaultCommitMessage = "git-xargs programmatic commit"
2120
DefaultPullRequestTitle = "git-xargs programmatic pull request"
@@ -81,11 +80,6 @@ var (
8180
Usage: "The description to add to pull requests opened by git-xargs",
8281
Value: DefaultPullRequestDescription,
8382
}
84-
GenericMaxConcurrentReposFlag = cli.IntFlag{
85-
Name: MaxConcurrentReposFlagName,
86-
Usage: "Limits the number of concurrent processed repositories. This is only useful if you encounter issues and need throttling when running on a very large number of repos. Default is 0 (Unlimited)",
87-
Value: DefaultMaxConcurrentRepos,
88-
}
8983
GenericSecondsToWaitFlag = cli.IntFlag{
9084
Name: SecondsToWaitBetweenPrsFlagName,
9185
Usage: "The number of seconds to sleep between pull requests in order to respect GitHub API rate limits. Increase this number if you are being rate limited regularly. Defaults to 12 seconds.",

data/test/good-test-repos.txt

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1 @@
1-
gruntwork-io/cloud-nuke,
2-
"gruntwork-io/fetch",
3-
gruntwork-io/bash-commons
1+
gruntwork-io/terragrunt

data/test/test-file-parsing.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
gruntwork-io/fetch,
2+
gruntwork-io/cloud-nuke,
3+
gruntwork-io/bash-commons

go.mod

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,7 @@ require (
99
github.com/gruntwork-io/go-commons v0.8.2
1010
github.com/kataras/tablewriter v0.0.0-20180708051242-e063d29b7c23
1111
github.com/landoop/tableprinter v0.0.0-20200805134727-ea32388e35c1
12-
github.com/mattn/go-runewidth v0.0.9 // indirect
13-
github.com/remeh/sizedwaitgroup v1.0.0
12+
github.com/pterm/pterm v0.12.42
1413
github.com/sirupsen/logrus v1.7.0
1514
github.com/stretchr/testify v1.7.0
1615
github.com/urfave/cli v1.22.5

0 commit comments

Comments
 (0)