Skip to content

feat: add mixed-precision and agc to gradaccum optimizer #131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open

feat: add mixed-precision and agc to gradaccum optimizer #131

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
8107142
feat: upgrade optimizer and agc
Jan 21, 2024
7e31c64
feat: swap base class for
Jan 21, 2024
0cb355a
fix: change to absolute imports for clarity and ease of testing
Jan 30, 2024
40e1f98
fix: install, lint, black
Feb 5, 2024
fde7729
revert: relative imports
Feb 5, 2024
f2680f7
revert: setup.py
Feb 5, 2024
18fcd31
revert: __init__.py
Feb 5, 2024
f239f3f
chore: linted
Feb 5, 2024
a9c2be5
revert: grad clipping, address in follow-up PR
Feb 5, 2024
96d6675
fix: uncomment broken unit test
Feb 5, 2024
609b7bb
chore: move to utils
Feb 5, 2024
858a574
chore: re-lint
Feb 6, 2024
311cc35
fix: failing unit test
Feb 6, 2024
8a0183f
test: update mixed_precision unit test to also test optimizer
Feb 6, 2024
3264e5e
fix: broken import (agc)
Feb 6, 2024
472af58
feat: revert test_mixed_precision
Feb 7, 2024
fa1ef2e
feat: increase test coverage
Feb 8, 2024
657b4f0
feat: cover parse_grad
Feb 8, 2024
e382e16
fix: missing var def
Feb 8, 2024
4263d20
chore: update codecov.yml
Feb 8, 2024
13111b2
fix: failing unit test
Feb 8, 2024
edc6221
test: update test_optimizer with additional coverage
Feb 11, 2024
8407d17
test: add support for gradients property, clip factor to agc
Feb 11, 2024
682b346
test: fix lr test
Feb 11, 2024
e97a43c
chore: lint
Feb 11, 2024
03d9076
fix: clip_factor -> clipvalue
Feb 11, 2024
da1c346
chore: lint
Feb 11, 2024
7188e30
chore: lint
Feb 11, 2024
8d87ab9
chore: lint
Feb 11, 2024
58e42e8
test(cov): lr setter
Feb 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/codecov.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,8 @@ jobs:
--cov=gradient_accumulator tests/test_mp_batch_norm.py \
--cov=gradient_accumulator tests/test_bn_convnd.py \
--cov=gradient_accumulator tests/test_bn_pretrained_swap.py \
--cov=gradient_accumulator tests/test_model_distribute.py
--cov=gradient_accumulator tests/test_model_distribute.py \
--cov=gradient_accumulator tests/test_optimizer.py

- name: Lint with flake8
run: flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
Expand Down
Loading