This repository was archived by the owner on May 1, 2025. It is now read-only.
This repository was archived by the owner on May 1, 2025. It is now read-only.
cannot reproduce output for custom dataset from example #50
Description
I am trying to reproduce the output using example inference code for custom dataset. Followed the shown process to generate my_custom_dataset but the output does not match.
Command:
docker run -it --rm -v pwd:/decaNLP/ -u $(id -u):$(id -g) bmccann/decanlp:torch041 bash -c "python /decaNLP/predict.py --evaluate valid --path /decaNLP/squad_mqan_cove_cpu --checkpoint_name iteration_39000.pth --tasks my_custom_dataset --device -1"
Here is the output I am getting, which does not show the prediction at all. It also keeps expanding the vocabulary everytime I run the above command.
39000
77.18
Arguments:
{'best_checkpoint': '/decaNLP/squad_mqan_cove_cpu/iteration_39000.pth',
'bleu': False,
'checkpoint_name': 'iteration_39000.pth',
'cove': True,
'data': '/decaNLP/.data/',
'devices': [-1],
'dimension': 200,
'dropout_ratio': 0.0,
'embeddings': '/decaNLP/.embeddings',
'evaluate': 'valid',
'intermediate_cove': False,
'load': None,
'lower': True,
'max_generative_vocab': 50000,
'max_output_length': 100,
'max_val_context_length': 400,
'model': 'MultitaskQuestionAnsweringNetwork',
'overwrite_predictions': False,
'path': '/decaNLP/squad_mqan_cove_cpu',
'rnn_layers': 1,
'rouge': False,
'seed': 123,
'silent': False,
'task_to_metric': {'cnn_dailymail': 'avg_rouge',
'iwslt.en.de': 'bleu',
'multinli.in.out': 'em',
'schema': 'em',
'squad': 'nf1',
'srl': 'nf1',
'sst': 'em',
'wikisql': 'lfem',
'woz.en': 'joint_goal_em',
'zre': 'corpus_f1'},
'tasks': ['my_custom_dataset'],
'transformer_heads': 3,
'transformer_hidden': 150,
'transformer_layers': 2,
'val_batch_size': [256]}
Loading from /decaNLP/squad_mqan_cove_cpu/iteration_39000.pth
Initializing Model
Loading my_custom_dataset
Caching data to /decaNLP/.data/my_custom_dataset/.cache/val.jsonl/None
Vocabulary has 87176 tokens from training
Vocabulary has expanded to 87176 tokens
100%|█████████████████████████████████████████████████████████████████████████████████████| 874474/874474 [00:30<00:00, 28815.11it/s]
I am running this on Macbook Pro with Anaconda. Here is the output for conda list:
# packages in environment at /Users/MY_USER_NAME/anaconda3:
#
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py36_0
alabaster 0.7.12 py36_0
anaconda 5.0.1 py36h6e48e2d_1
anaconda-client 1.7.2 py36_0
anaconda-navigator 1.9.2 py36_0
anaconda-project 0.8.2 py36_0
appnope 0.1.0 py36hf537a9a_0
appscript 1.0.1 py36h1de35cc_1
asn1crypto 0.24.0 py36_0
astroid 2.1.0 py36_0
astropy 3.0.5 py36h1de35cc_0
atomicwrites 1.2.1 py36_0
attrs 18.2.0 py36h28b3542_0
babel 2.6.0 py36_0
backcall 0.1.0 py36_0
backports 1.0 py36_1
backports.os 0.1.1 py36_0
backports.shutil_get_terminal_size 1.0.0 py36_2
beautifulsoup4 4.6.3 py36_0
bitarray 0.8.3 py36h1de35cc_0
bkcharts 0.2 py36h073222e_0
blas 1.0 mkl
blaze 0.11.3 py36h02e7a37_0
bleach 3.0.2 py36_0
blosc 1.14.4 hd9629dc_0
bokeh 1.0.2 py36_0
boto 2.49.0 py36_0
bottleneck 1.2.1 py36h1d22016_1
bzip2 1.0.6 h1de35cc_5
ca-certificates 2019.1.23 0
certifi 2018.11.29 py36_0
cffi 1.11.5 py36h6174b99_1
chardet 3.0.4 py36_1
click 7.0 py36_0
cloudpickle 0.6.1 py36_0
clyent 1.2.2 py36_1
colorama 0.4.0 py36_0
conda 4.6.3 py36_0
conda-build 3.17.1 py36_0
conda-env 2.6.0 1
conda-verify 3.1.1 py36_0
contextlib2 0.5.5 py36hd66e5e7_0
cryptography 2.5 py36ha12b0ac_0
curl 7.63.0 ha441bb4_1000
cycler 0.10.0 py36hfc81398_0
cython 0.29 py36h0a44026_0
cytoolz 0.9.0.1 py36h1de35cc_1
dask 1.0.0 py36_0
dask-core 1.0.0 py36_0
datashape 0.5.4 py36_1
dbus 1.13.2 h760590f_1
decorator 4.3.0 py36_0
distributed 1.25.0 py36_0
docutils 0.14 py36hbfde631_0
entrypoints 0.2.3 py36_2
et_xmlfile 1.0.1 py36h1315bdc_0
expat 2.2.6 h0a44026_0
fastcache 1.0.2 py36h1de35cc_2
filelock 3.0.10 py36_0
flake8 3.6.0 py36_0
flask 1.0.2 py36_1
flask-cors 3.0.7 py36_0
freetype 2.9.1 hb4e5f40_0
future 0.17.1 py36_0
get_terminal_size 1.0.0 h7520d66_0
gettext 0.19.8.1 h15daf44_3
gevent 1.3.7 py36h1de35cc_1
glib 2.56.2 hd9629dc_0
glob2 0.6 py36_1
gmp 6.1.2 hb37e062_1
gmpy2 2.0.8 py36h6ef4df4_2
greenlet 0.4.15 py36h1de35cc_0
h5py 2.8.0 py36h878fce3_3
hdf5 1.10.2 hfa1e0ec_1
heapdict 1.0.0 py36_2
html5lib 1.0.1 py36_0
icu 58.2 h4b95b61_1
idna 2.7 py36_0
imageio 2.4.1 py36_0
imagesize 1.1.0 py36_0
importlib_metadata 0.6 py36_0
intel-openmp 2019.1 144
ipykernel 5.1.0 py36h39e3cac_0
ipython 7.2.0 py36h39e3cac_0
ipython_genutils 0.2.0 py36h241746c_0
ipywidgets 7.4.2 py36_0
isort 4.3.4 py36_0
itsdangerous 1.1.0 py36_0
jbig 2.1 h4d881f8_0
jdcal 1.4 py36_0
jedi 0.13.1 py36_0
jinja2 2.10 py36_0
jpeg 9b he5867d9_2
jsonschema 2.6.0 py36hb385e00_0
jupyter 1.0.0 py36_7
jupyter_client 5.2.3 py36_0
jupyter_console 6.0.0 py36_0
jupyter_core 4.4.0 py36_0
jupyterlab 0.35.3 py36_0
jupyterlab_launcher 0.13.1 py36_0
jupyterlab_server 0.2.0 py36_0
keyring 16.1.1 py36_0
kiwisolver 1.0.1 py36h0a44026_0
krb5 1.16.1 hddcf347_7
lazy-object-proxy 1.3.1 py36h1de35cc_2
libarchive 3.3.3 he8b1da1_2
libcurl 7.63.0 h051b688_1000
libcxx 4.0.1 hcfea43d_1
libcxxabi 4.0.1 hcfea43d_1
libedit 3.1.20170329 hb402a30_2
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
liblief 0.9.0 h2a1bed3_0
libpng 1.6.35 ha441bb4_0
libsodium 1.0.16 h3efe00b_0
libssh2 1.8.0 ha12b0ac_4
libtiff 4.0.9 hcb84e12_2
libxml2 2.9.8 hab757c2_1
libxslt 1.1.32 hb819dd2_0
llvmlite 0.26.0 py36h8c7ce04_0
locket 0.2.0 py36hca03003_1
lxml 4.2.5 py36hef8c89e_0
lz4-c 1.8.1.2 h1de35cc_0
lzo 2.10 h362108e_2
markupsafe 1.1.0 py36h1de35cc_0
matplotlib 3.0.1 py36h54f8f79_0
mccabe 0.6.1 py36_1
mistune 0.8.4 py36h1de35cc_0
mkl 2018.0.3 1
mkl-service 1.1.2 py36h6b9c3cc_5
mkl_fft 1.0.6 py36hb8a8100_0
mkl_random 1.0.1 py36h5d10147_1
more-itertools 4.3.0 py36_0
mpc 1.1.0 h6ef4df4_1
mpfr 4.0.1 h3018a27_3
mpmath 1.0.0 py36_2
msgpack-python 0.5.6 py36h04f5b5a_1
multipledispatch 0.6.0 py36_0
navigator-updater 0.2.1 py36_0
nbconvert 5.3.1 py36_0
nbformat 4.4.0 py36h827af21_0
ncurses 6.1 h0a44026_1
networkx 2.2 py36_1
ninja 1.8.2 py36h04f5b5a_1
nltk 3.3.0 py36_0
nose 1.3.7 py36_2
notebook 5.7.2 py36_0
numba 0.35.0 np113py36_6
numexpr 2.6.8 py36h1dc9127_0
numpy 1.15.4 py36h6a91979_0
numpy-base 1.15.4 py36h8a80b8c_0
numpydoc 0.8.0 py36_0
odo 0.5.1 py36hc1af34a_0
olefile 0.46 py36_0
openpyxl 2.5.11 py36_0
openssl 1.1.1a h1de35cc_0
packaging 18.0 py36_0
pandas 0.23.4 py36h6440ff4_0
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py36_1
parso 0.3.1 py36_0
partd 0.3.9 py36_0
path.py 11.5.0 py36_0
pathlib2 2.3.2 py36_0
patsy 0.5.1 py36_0
pcre 8.42 h378b8a2_0
pep8 1.7.1 py36_0
pexpect 4.6.0 py36_0
pickleshare 0.7.5 py36_0
pillow 5.3.0 py36hb68e598_0
pip 18.1 py36_0
pkginfo 1.4.2 py36_1
pluggy 0.8.0 py36_0
ply 3.11 py36_0
prometheus_client 0.4.2 py36_0
prompt_toolkit 2.0.7 py36_0
psutil 5.4.8 py36h1de35cc_0
ptyprocess 0.6.0 py36_0
py 1.7.0 py36_0
py-lief 0.9.0 py36hd4eaf27_0
pycodestyle 2.4.0 py36_0
pycosat 0.6.3 py36h1de35cc_0
pycparser 2.19 py36_0
pycrypto 2.6.1 py36h1de35cc_9
pycurl 7.43.0.2 py36ha12b0ac_0
pyflakes 2.0.0 py36_0
pygments 2.2.0 py36h240cd3f_0
pylint 2.2.2 py36_0
pyodbc 4.0.24 py36h0a44026_0
pyopenssl 18.0.0 py36_0
pyparsing 2.3.0 py36_0
pyqt 5.9.2 py36h655552a_2
pysocks 1.6.8 py36_0
pytables 3.4.4 py36h13cba08_0
pytest 4.0.1 py36_0
pytest-arraydiff 0.2 py36h39e3cac_0
pytest-astropy 0.4.0 py36_0
pytest-doctestplus 0.2.0 py36_0
pytest-openfiles 0.3.1 py36_0
pytest-remotedata 0.3.1 py36_0
python 3.6.7 haf84260_0
python-dateutil 2.7.5 py36_0
python-libarchive-c 2.8 py36_6
python.app 2 py36_9
pytorch 0.4.1 py36_cuda0.0_cudnn0.0_1 pytorch
pytz 2018.7 py36_0
pywavelets 1.0.1 py36h1d22016_0
pyyaml 3.13 py36h1de35cc_0
pyzmq 17.1.2 py36h1de35cc_0
qt 5.9.6 h45cd832_2
qtawesome 0.5.3 py36_0
qtconsole 4.4.2 py36_0
qtpy 1.5.2 py36_0
readline 7.0 h1de35cc_5
requests 2.20.1 py36_0
rope 0.11.0 py36_0
ruamel_yaml 0.15.46 py36h1de35cc_0
scikit-image 0.14.0 py36h0a44026_1
scikit-learn 0.20.1 py36h4f467ca_0
scipy 1.1.0 py36h28f7352_1
seaborn 0.9.0 py36_0
send2trash 1.5.0 py36_0
setuptools 40.6.2 py36_0
simplegeneric 0.8.1 py36_2
singledispatch 3.4.0.3 py36hf20db9d_0
sip 4.19.8 py36h0a44026_0
six 1.11.0 py36_1
snappy 1.1.7 he62c110_3
snowballstemmer 1.2.1 py36h6c7b616_0
sortedcollections 1.0.1 py36_0
sortedcontainers 2.1.0 py36_0
sphinx 1.8.2 py36_0
sphinxcontrib 1.0 py36_1
sphinxcontrib-websupport 1.1.0 py36_1
spyder 3.3.2 py36_0
spyder-kernels 0.3.0 py36_0
sqlalchemy 1.2.14 py36h1de35cc_0
sqlite 3.25.3 ha441bb4_0
statsmodels 0.9.0 py36h1d22016_0
sympy 1.3 py36_0
tblib 1.3.2 py36hda67792_0
terminado 0.8.1 py36_1
testpath 0.4.2 py36_0
tk 8.6.8 ha441bb4_0
toolz 0.9.0 py36_0
torchvision 0.2.1 py36_1 pytorch
tornado 5.1.1 py36h1de35cc_0
tqdm 4.28.1 py36h28b3542_0
traitlets 4.3.2 py36h65bd3ce_0
typed-ast 1.1.0 py36h1de35cc_0
typing 3.6.4 py36_0
unicodecsv 0.14.1 py36he531d66_0
unixodbc 2.3.7 h1de35cc_0
urllib3 1.23 py36_0
wcwidth 0.1.7 py36h8c6ec74_0
webencodings 0.5.1 py36_1
werkzeug 0.14.1 py36_0
wheel 0.32.3 py36_0
widgetsnbextension 3.4.2 py36_0
wrapt 1.10.11 py36h1de35cc_2
wurlitzer 1.0.2 py36_0
xlrd 1.1.0 py36_1
xlsxwriter 1.1.2 py36_0
xlwings 0.15.0 py36_0
xlwt 1.2.0 py36h5ad1178_0
xz 5.2.4 h1de35cc_4
yaml 0.1.7 hc338f04_2
zeromq 4.2.5 h0a44026_1
zict 0.1.3 py36_0
zlib 1.2.11 h1de35cc_3
zstd 1.3.3 h2a6be3a_0