Project: Virtual Private Cloud (#345)

* Add support for VPCs

* Drop debug statement

* Clean up

* Revert

* oops

* Add docs

* Add reorder test

* Add reorder test

* make format

* oops

* Fix VPCSubnet docs

* Address feedback

* Use `asdict` to convert dataclass objects to dicts (#324)

* test: additional integration tests for vpc (#335)

## 📝 Description

- Extending test coverage for VPC 

## ✔️ How to Test

1. Setup API token for alpha/beta environment and export it
LINODE_TOKEN=mytoken

2. get cacert.pem (e.g. wget https://certurl.com/cacert.pem)

3. make slight modification to `def get_client()` in conftest.py
e.g. `client = LinodeClient(token,
base_url='https://api.dev.linode.com/v4beta',
ca_path='/Users/ykim/linode/ykim/linode_api4-python/cacert.pem')`

4. run test `pytest test/integration/models/test_vpc.py`


## 📷 Preview

**If applicable, include a screenshot or code snippet of this change.
Otherwise, please remove this section.**

---------

Co-authored-by: Zhiwei Liang <[email protected]>
Co-authored-by: Zhiwei Liang <[email protected]>

* Add `vpc_nat_1_1` to IPAddress (#342)

## 📝 Description

This change is adding `vpc_nat_1_1` to the IPAddress for VPC. If a
public IPv4 address is NAT 1:1 mapped to a private VPC IP, this field is
returned VPC IP together with the VPC and subnet ids.

Also trying to merge two commits from dev to proj/vpc to update the
feature branch. The actual change is focusing on
3206ab8.

## ✔️ How to Test

Build unit test to make sure that we can retrieve this field from
IPAddress object:
`tox`

---------

Co-authored-by: Zhiwei Liang <[email protected]>

* implementation

* fixing tests

* json_object

* Support list deserialise in `JSONObject`

* cleaning up code

* make format

* Replace `get_client` with `test_linode_client` (#346)

* fix: Handle `null` values in `JSONObject` fields (#344)

## 📝 Description

**What does this PR do and why is this change necessary?**

Handle null values for `JSONObject` fields

## ✔️ How to Test

**What are the steps to reproduce the issue or verify the changes?**

`pytest test/integration/models/test_networking.py`

---------

Co-authored-by: Lena Garber <[email protected]>
Co-authored-by: Lena Garber <[email protected]>
Co-authored-by: Youjung Kim <[email protected]>
Co-authored-by: Ye Chen <[email protected]>
Co-authored-by: Ania Misiorek <[email protected]>
Co-authored-by: Ania Misiorek <[email protected]>
Co-authored-by: Jacob Riddle <[email protected]>

Author: 6 people

linode_api4/groups/__init__.py

@@ -18,3 +18,4 @@
 from .support import *
 from .tag import *
 from .volume import *
+from .vpc import *

linode_api4/groups/linode.py

@@ -8,6 +8,8 @@
 from linode_api4.objects import (
     AuthorizedApp,
     Base,
+    ConfigInterface,
+    Firewall,
     Image,
     Instance,
     Kernel,
@@ -250,6 +252,8 @@ def instance_create(
                          The contents of this field can be built using the
                          :any:`build_instance_metadata` method.
         :type metadata: dict
+        :param firewall: The firewall to attach this Linode to.
+        :type firewall: int or Firewall
 
         :returns: A new Instance object, or a tuple containing the new Instance and
                   the generated password.
@@ -284,6 +288,10 @@ def instance_create(
             )
             del kwargs["backup"]
 
+        if "firewall" in kwargs:
+            fw = kwargs.pop("firewall")
+            kwargs["firewall_id"] = fw.id if isinstance(fw, Firewall) else fw
+
         params = {
             "type": ltype.id if issubclass(type(ltype), Base) else ltype,
             "region": region.id if issubclass(type(region), Base) else region,
@@ -292,6 +300,7 @@ def instance_create(
             else None,
             "authorized_keys": authorized_keys,
         }
+
         params.update(kwargs)
 
         result = self.client.post("/linode/instances", data=params)

linode_api4/groups/vpc.py

@@ -0,0 +1,83 @@
+from typing import Any, Dict, List, Optional, Union
+
+from linode_api4 import VPCSubnet
+from linode_api4.errors import UnexpectedResponseError
+from linode_api4.groups import Group
+from linode_api4.objects import VPC, Base, Region
+from linode_api4.paginated_list import PaginatedList
+
+
+class VPCGroup(Group):
+    def __call__(self, *filters) -> PaginatedList:
+        """
+        Retrieves all of the VPCs the acting user has access to.
+
+        This is intended to be called off of the :any:`LinodeClient`
+        class, like this::
+
+           vpcs = client.vpcs()
+
+        API Documentation: TODO
+
+        :param filters: Any number of filters to apply to this query.
+                        See :doc:`Filtering Collections</linode_api4/objects/filtering>`
+                        for more details on filtering.
+
+        :returns: A list of VPC the acting user can access.
+        :rtype: PaginatedList of VPC
+        """
+        return self.client._get_and_filter(VPC, *filters)
+
+    def create(
+        self,
+        label: str,
+        region: Union[Region, str],
+        description: Optional[str] = None,
+        subnets: Optional[List[Dict[str, Any]]] = None,
+        **kwargs,
+    ) -> VPC:
+        """
+        Creates a new VPC under your Linode account.
+
+        API Documentation: TODO
+
+        :param label: The label of the newly created VPC.
+        :type label: str
+        :param region: The region of the newly created VPC.
+        :type region: Union[Region, str]
+        :param description: The user-defined description of this VPC.
+        :type description: Optional[str]
+        :param subnets: A list of subnets to create under this VPC.
+        :type subnets: List[Dict[str, Any]]
+
+        :returns: The new VPC object.
+        :rtype: VPC
+        """
+        params = {
+            "label": label,
+            "region": region.id if isinstance(region, Region) else region,
+        }
+
+        if description is not None:
+            params["description"] = description
+
+        if subnets is not None and len(subnets) > 0:
+            for subnet in subnets:
+                if not isinstance(subnet, dict):
+                    raise ValueError(
+                        f"Unsupported type for subnet: {type(subnet)}"
+                    )
+
+            params["subnets"] = subnets
+
+        params.update(kwargs)
+
+        result = self.client.post("/vpcs", data=params)
+
+        if not "id" in result:
+            raise UnexpectedResponseError(
+                "Unexpected response when creating VPC", json=result
+            )
+
+        d = VPC(self.client, result["id"], result)
+        return d

linode_api4/linode_client.py

@@ -28,6 +28,7 @@
     SupportGroup,
     TagGroup,
     VolumeGroup,
+    VPCGroup,
 )
 from linode_api4.objects import Image, and_
 from linode_api4.objects.filtering import Filter
@@ -190,6 +191,9 @@ def __init__(
         #: Access methods related to Images - See :any:`ImageGroup` for more information.
         self.images = ImageGroup(self)
 
+        #: Access methods related to VPCs - See :any:`VPCGroup` for more information.
+        self.vpcs = VPCGroup(self)
+
         #: Access methods related to Event polling - See :any:`PollingGroup` for more information.
         self.polling = PollingGroup(self)
 

linode_api4/objects/__init__.py

@@ -1,6 +1,7 @@
 # isort: skip_file
 from .base import Base, Property, MappedObject, DATE_FORMAT, ExplicitNullValue
 from .dbase import DerivedBase
+from .serializable import JSONObject
 from .filtering import and_, or_
 from .region import Region
 from .image import Image
@@ -17,4 +18,5 @@
 from .object_storage import *
 from .lke import *
 from .database import *
+from .vpc import *
 from .beta import *

linode_api4/objects/base.py

@@ -1,6 +1,8 @@
 import time
 from datetime import datetime, timedelta
 
+from linode_api4.objects.serializable import JSONObject
+
 from .filtering import FilterableMetaclass
 
 DATE_FORMAT = "%Y-%m-%dT%H:%M:%S"
@@ -30,6 +32,7 @@ def __init__(
         id_relationship=False,
         slug_relationship=False,
         nullable=False,
+        json_object=None,
     ):
         """
         A Property is an attribute returned from the API, and defines metadata
@@ -56,6 +59,8 @@ def __init__(
         self.is_datetime = is_datetime
         self.id_relationship = id_relationship
         self.slug_relationship = slug_relationship
+        self.nullable = nullable
+        self.json_class = json_object
 
 
 class MappedObject:
@@ -111,7 +116,7 @@ class Base(object, metaclass=FilterableMetaclass):
 
     properties = {}
 
-    def __init__(self, client, id, json={}):
+    def __init__(self, client: object, id: object, json: object = {}) -> object:
         self._set("_populated", False)
         self._set("_last_updated", datetime.min)
         self._set("_client", client)
@@ -123,8 +128,8 @@ def __init__(self, client, id, json={}):
         #: be updated on access.
         self._set("_raw_json", None)
 
-        for prop in type(self).properties:
-            self._set(prop, None)
+        for k in type(self).properties:
+            self._set(k, None)
 
         self._set("id", id)
         if hasattr(type(self), "id_attribute"):
@@ -289,7 +294,7 @@ def _serialize(self):
 
             value = getattr(self, k)
 
-            if not value:
+            if not v.nullable and (value is None or value == ""):
                 continue
 
             # Let's allow explicit null values as both classes and instances
@@ -305,7 +310,7 @@ def _serialize(self):
         for k, v in result.items():
             if isinstance(v, Base):
                 result[k] = v.id
-            elif isinstance(v, MappedObject):
+            elif isinstance(v, MappedObject) or issubclass(type(v), JSONObject):
                 result[k] = v.dict
 
         return result
@@ -376,6 +381,18 @@ def _populate(self, json):
                         .properties[key]
                         .slug_relationship(self._client, json[key]),
                     )
+                elif type(self).properties[key].json_class:
+                    json_class = type(self).properties[key].json_class
+                    json_value = json[key]
+
+                    # build JSON object
+                    if isinstance(json_value, list):
+                        # We need special handling for list responses
+                        value = [json_class.from_json(v) for v in json_value]
+                    else:
+                        value = json_class.from_json(json_value)
+
+                    self._set(key, value)
                 elif type(json[key]) is dict:
                     self._set(key, MappedObject(**json[key]))
                 elif type(json[key]) is list: