Feature/Update default language models to UD 2.5 (#12)

* Refactor code (mostly static type hints)

* Update default models to latest versions (UD 2.5)

* Update LICENSE, setup.py, and README.md

* Add more tests

Author: asajatovic

.github/workflows/pythonpackage.yml

@@ -32,7 +32,11 @@ jobs:
         flake8 **/*.py --count --show-source --statistics
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
         flake8 **/*.py --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+    - name: Install package
+      run: |
+        # required because of entry points
+        pip install .
     - name: Test with pytest
       run: |
         pip install pytest
-        python -m pytest -vvv test
+        python -m pytest -vvv tests

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2019 Text Analysis and Knowledge Engineering Lab (TakeLab)
+Copyright (c) 2019-2020 Text Analysis and Knowledge Engineering Lab (TakeLab)
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -2,9 +2,9 @@
 
 This package wraps the fast and efficient [UDPipe](http://ufal.mff.cuni.cz/udpipe) language-agnostic NLP pipeline
 (via its [Python bindings](https://github.com/ufal/udpipe/tree/master/bindings/python)), so you can use
-[UDPipe pre-trained models](https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-2998) as a [spaCy](https://spacy.io/) pipeline for 50+ languages out-of-the-box.
-Inspired by [spacy-stanfordnlp](https://github.com/explosion/spacy-stanfordnlp), this package offers slightly less accurate
-models that are in turn much faster (see benchmarks for [UDPipe](https://ufal.mff.cuni.cz/udpipe/models#universal_dependencies_24_models_performance) and [StanfordNLP](https://stanfordnlp.github.io/stanfordnlp/performance.html)).
+[UDPipe pre-trained models](https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-3131) as a [spaCy](https://spacy.io/) pipeline for 50+ languages out-of-the-box.
+Inspired by [spacy-stanza](https://github.com/explosion/spacy-stanza), this package offers slightly less accurate
+models that are in turn much faster (see benchmarks for [UDPipe](https://ufal.mff.cuni.cz/udpipe/models#universal_dependencies_25_models_performance) and [Stanza](https://stanfordnlp.github.io/stanza/performance.html)).
 
 ## Installation
 
@@ -14,10 +14,10 @@ Use the package manager [pip](https://pip.pypa.io/en/stable/) to install spacy-u
 pip install spacy-udpipe
 ```
 
-After installation, use `spacy_udpipe.download(<language ISO code>)` to download the pre-trained model for the desired language.
+After installation, use `spacy_udpipe.download()` to download the pre-trained model for the desired language.
 
 ## Usage
-The loaded UDPipeLanguage class returns a spaCy [`Language` object](https://spacy.io/api/language), i.e., the nlp object you can use to process text and create a [`Doc` object](https://spacy.io/api/doc).
+The loaded UDPipeLanguage class returns a spaCy [`Language` object](https://spacy.io/api/language), i.e., the object you can use to process text and create a [`Doc` object](https://spacy.io/api/doc).
 
 ```python
 import spacy_udpipe
@@ -56,18 +56,19 @@ Created by [Antonio Šajatović](http://github.com/asajatovic) during an interns
 ## Contributing
 Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
 
-Please make sure to update tests as appropriate. Tests are run automatically for each pull request on the master branch. To start the tests locally, just run [`pytest`](https://docs.pytest.org/en/latest/contents.html) in the root source directory.
+Please make sure to update the tests as appropriate. Tests are run automatically for each pull request on the master branch.
+To start the tests locally, first, install the package with `pip install -e .`, then run [`pytest`](https://docs.pytest.org/en/latest/contents.html) in the root source directory.
 
 ## License
-[MIT](https://choosealicense.com/licenses/mit/)  © Text Analysis and Knowledge Engineering Lab (TakeLab)
+[MIT](https://choosealicense.com/licenses/mit/) © Text Analysis and Knowledge Engineering Lab (TakeLab)
 
 ## Project status
 Maintained by [Text Analysis and Knowledge Engineering Lab (TakeLab)](http://takelab.fer.hr/).
 
 ## Notes
 * All available pre-trained models are licensed under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).
 
-* Full list of pre-trained models for supported languages is available in [`languages.json`](https://github.com/TakeLab/spacy-udpipe/blob/master/spacy_udpipe/languages.json).
+* A full list of pre-trained models for supported languages is available in [`languages.json`](https://github.com/TakeLab/spacy-udpipe/blob/master/spacy_udpipe/languages.json).
 
 * This package exposes a `spacy_languages` entry point in its [`setup.py`](https://github.com/TakeLab/spacy-udpipe/blob/master/setup.py) so full suport for serialization is enabled:
     ```python
@@ -84,10 +85,10 @@ Maintained by [Text Analysis and Knowledge Engineering Lab (TakeLab)](http://tak
 * Known possible issues:
     * Tag map
 
-      `Token.tag_` is a [CoNLL](https://universaldependencies.org/format.html) XPOS tag (language-specific part-of-speech tag), defined for each language separately by the corresponding [Universal Dependencies](https://universaldependencies.org/) treebank. Mappings between between XPOS and Universal Dependencies POS tags should be defined in a `TAG_MAP` dictionary (located in language-specific `tag_map.py` files), along with optional morphological features. See [spaCy tag map](https://spacy.io/usage/adding-languages#tag-map) for more details.
+      `Token.tag_` is a [CoNLL](https://universaldependencies.org/format.html) XPOS tag (language-specific part-of-speech tag), defined for each language separately by the corresponding [Universal Dependencies](https://universaldependencies.org/) treebank. Mappings between XPOS and Universal Dependencies POS tags should be defined in a `TAG_MAP` dictionary (located in language-specific `tag_map.py` files), along with optional morphological features. See [spaCy tag map](https://spacy.io/usage/adding-languages#tag-map) for more details.
     * Syntax iterators
 
       In order to extract `Doc.noun_chunks`, a proper syntax iterator implementation for the language of interest is required. For more details, please see [spaCy syntax iterators](https://spacy.io/usage/adding-languages#syntax-iterators).
     * Other language-specific issues
 
-      A quick way to check language-specific defaults in [spaCy](https://spacy.io) is to visit [spaCy language support](https://spacy.io/usage/models#languages). Also, please see [spaCy language data](https://spacy.io/usage/adding-languages#language-data) for details regarding other language-specific data.
+      A quick way to check language-specific defaults in [spaCy](https://spacy.io) is to visit [spaCy language support](https://spacy.io/usage/models#languages). Also, please see [spaCy language data](https://spacy.io/usage/adding-languages#language-data) for details regarding other language-specific data.

setup.py

@@ -3,41 +3,44 @@
 
 import setuptools
 
-with open("README.md", "r") as fh:
-    long_description = fh.read()
-
 URL = "https://github.com/TakeLab/spacy-udpipe"
 
-# get a dict of available languages from languages.json
-root = os.path.abspath(os.path.dirname(__file__))
-langs_path = os.path.join(root, "spacy_udpipe", "languages.json")
-with open(langs_path, "r") as f:
-    LANGUAGES = json.load(f)
+with open("README.md", "r") as f:
+    readme = f.read()
+
+# Get available languages and models from spacy_udpipe/languages.json
+languages_path = os.path.join(
+    os.path.abspath(os.path.dirname(__file__)),
+    "spacy_udpipe",
+    "languages.json"
+)
+with open(languages_path, "r") as f:
+    languages = json.load(f)
 
-ENTRY_LANGS = set(f"udpipe_{s.split('-')[0]} = spacy_udpipe:UDPipeLanguage"
-                  for s in LANGUAGES.keys())
+ENTRY_POINTS = {"spacy_languages":
+                set(f"udpipe_{s.split('-')[0]} = "
+                    "spacy_udpipe:UDPipeLanguage"
+                    for s in languages.keys()
+                    )
+                }
 
 setuptools.setup(
-    name="spacy-udpipe",
-    version="0.1.0",
+    name="spacy_udpipe",
+    version="0.2.0",
     description="Use fast UDPipe models directly in spaCy",
-    long_description=long_description,
+    long_description=readme,
     long_description_content_type="text/markdown",
     url=URL,
     author="TakeLab",
     author_email="[email protected]",
-    license='MIT',
-    keywords='udpipe spacy nlp',
+    license="MIT",
+    keywords="nlp udpipe spacy python",
     packages=setuptools.find_packages(),
     install_requires=["spacy>=2.1.0", "ufal.udpipe>=1.2.0"],
     python_requires=">=3.6",
-    entry_points={
-        "spacy_languages": ENTRY_LANGS
-    },
+    entry_points=ENTRY_POINTS,
     tests_require=["pytest>=5.0.0"],
-    package_data={
-        'spacy_udpipe': ['./languages.json'],
-    },
+    package_data={"spacy_udpipe": ["./languages.json"], },
     classifiers=[
         "Development Status :: 4 - Beta",
         "Intended Audience :: Developers",
@@ -46,10 +49,11 @@
         "Operating System :: OS Independent",
     ],
     project_urls={
-        'SpaCy': 'https://spacy.io/',
-        'TakeLab': 'http://takelab.fer.hr/',
-        'UDPipe': 'http://ufal.mff.cuni.cz/udpipe',
-        'Source': URL,
-        'Tracker': URL + '/issues',
-    }
+        "SpaCy": "https://spacy.io/",
+        "TakeLab": "http://takelab.fer.hr/",
+        "UDPipe": "http://ufal.mff.cuni.cz/udpipe",
+        "Source": URL,
+        "Tracker": URL + "/issues",
+    },
+    zip_safe=False
 )

spacy_udpipe/__init__.py

@@ -1,5 +1,5 @@
 from .language import UDPipeLanguage, UDPipeModel, load, load_from_path
-from .util import download
+from .utils import download
 
 __all__ = ["UDPipeLanguage", "UDPipeModel",
            "load", "load_from_path", "download"]