[JShellAPI][Testing] Setting up First Integration Test - Simple code snippet evaluation scenario

JShellAPI/README.MD

@@ -101,4 +101,97 @@ The maximum ram allocated per container, in megabytes.
 ### jshellapi.dockerCPUsUsage
 The cpu configuration of each container, see [--cpus option of docker](https://docs.docker.com/config/containers/resource_constraints/#cpu).
 ### jshellapi.schedulerSessionKillScanRate
-The rate at which the session killer will check and delete session, in seconds, see [Session timeout](#Session-timeout).
+The rate at which the session killer will check and delete session, in seconds, see [Session timeout](#Session-timeout).
+
+## Testing
+
+>  The work on testing was made in collaboration with [Alathreon](https://github.com/Alathreon) and [Wazei](https://github.com/tj-wazei). I'd like thank both of them for their trust. - FirasRG
+
+This section outlines the work done to set up the first integration test that evaluates Java code by running it in a [Docker](https://www.docker.com/get-started/) container. The test ensures that the [Eval endpoint](#eval) can execute code within the containerized environment of [**JShellWrapper**](../JShellWrapper).
+
+### Usage
+
+```java
+@ContextConfiguration(classes = Main.class)
+@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
+public class JShellApiTests {
+
+    @Autowired
+    private WebTestClient webTestClient;
+
+    @Test
+    @DisplayName("When posting code snippet, evaluate it then returns successfully result")
+    public void evaluateCodeSnippetTest() {
+
+        final String testEvalId = "test";
+
+        final String firstCodeExpression = "int a = 2+2;";
+
+        final JShellSnippetResult firstCodeSnippet = new JShellSnippetResult(SnippetStatus.VALID, SnippetType.ADDITION, 1, firstCodeExpression, "4");
+
+        final JShellResult firstCodeExpectedResult = getJShellResultDefaultInstance(firstCodeSnippet);
+
+        assertThat(testEval(testEvalId, firstCodeExpression)).isEqualTo(firstCodeExpectedResult);
+
+        // performing a second code execution test ...
+    }
+    // some methods ...
+}
+```
+
+### 1. Java Test Setup
+
+The [@SpringBootTest](https://docs.spring.io/spring-boot/api/java/org/springframework/boot/test/context/SpringBootTest.html) and [@ContextConfiguration](https://docs.spring.io/spring-framework/reference/testing/annotations/integration-spring/annotation-contextconfiguration.html) annotations are needed to prepare the app to tests, like in a real scenario.
+
+NOTE: _Test classes must be located under `/src/test/java/{org.togetherjava.jshellapi}`._
+
+- The test uses [WebTestClient](https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/test/web/reactive/server/WebTestClient.html) to make HTTP calls to the target endpoint.
+- Multiple API calls are made within the test method, so a utility instance method was created for reuse.
+- The test ensures that code is correctly evaluated inside the **JShellWrapper** container.
+
+### 2. Gradle Configuration for Tests
+
+The `build.gradle` of this project has been updated to handle **JShellWrapper** Docker image lifecycle during tests.
+
+- **JShellWrapper Image Name**: the image name is injected from the root [build.gradle](../build.gradle) file, to this project's [build.gradle](build.gradle) file and also to [application.yaml](src/main/resources/application.yaml)!
+- **JShellWrapper Docker Image**: The image is built before the tests run.
+- **Container & Cleanup**: After the tests finish, the container and image are removed to ensure a clean environment.
+
+```groovy
+def jshellWrapperImageName = rootProject.ext.jShellWrapperImageName;
+
+processResources {
+  filesMatching('application.yaml') {
+    expand(jShellWrapperImageName: jshellWrapperImageName)
+  }
+}
+
+def taskBuildDockerImage = tasks.register('buildDockerImage') {
+    group = 'docker'
+    description = 'builds jshellwrapper as docker image'
+    dependsOn project(':JShellWrapper').tasks.named('jibDockerBuild')
+}
+
+def taskRemoveDockerImage = tasks.register('removeDockerImage', Exec) {
+    group = 'docker'
+    description = 'removes jshellwrapper image'
+    commandLine 'docker', 'rmi', '-f', jshellWrapperImageName
+}
+
+test {
+    dependsOn taskBuildDockerImage
+    finalizedBy taskRemoveDockerImage
+}
+```
+
+Below are the key dependencies that were added or modified in the `build.gradle` file of this project :
+
+```groovy
+testImplementation('org.springframework.boot:spring-boot-starter-test') {
+  exclude group: 'ch.qos.logback', module: 'logback-classic'
+}
+testImplementation 'org.springframework.boot:spring-boot-starter-webflux'
+```
+
+- The `logback-classic` has been excluded because of an issue encountered when running tests. The issue is typically about a conflict between some dependencies (This solution has been brought based on [a _good_ answer on Stackoverflow](https://stackoverflow.com/a/42641450/10000150))
+- The `spring-boot-starter-webflux` was needed in order to be able to use **WebTestClient**.

JShellAPI/build.gradle

@@ -13,8 +13,13 @@ dependencies {
     implementation 'com.github.docker-java:docker-java-transport-httpclient5:3.3.6'
     implementation 'com.github.docker-java:docker-java-core:3.3.6'
 
-    testImplementation 'org.springframework.boot:spring-boot-starter-test'
+    testImplementation('org.springframework.boot:spring-boot-starter-test') {
+        exclude group: 'ch.qos.logback', module: 'logback-classic'
+    }
+    testImplementation 'org.springframework.boot:spring-boot-starter-webflux'
+
     annotationProcessor "org.springframework.boot:spring-boot-configuration-processor"
+
 }
 
 jib {
@@ -36,4 +41,30 @@ shadowJar {
     archiveBaseName.set('JShellPlaygroundBackend')
     archiveClassifier.set('')
     archiveVersion.set('')
-}
+}
+
+def jshellWrapperImageName = rootProject.ext.jShellWrapperImageName;
+
+processResources {
+    filesMatching('application.yaml') {
+        expand(jShellWrapperImageName: jshellWrapperImageName)
+    }
+}
+
+
+def taskBuildDockerImage = tasks.register('buildDockerImage') {
+    group = 'docker'
+    description = 'builds jshellwrapper as docker image'
+    dependsOn project(':JShellWrapper').tasks.named('jibDockerBuild')
+}
+
+def taskRemoveDockerImage = tasks.register('removeDockerImage', Exec) {
+    group = 'docker'
+    description = 'removes jshellwrapper image'
+    commandLine 'docker', 'rmi', '-f', jshellWrapperImageName
+}
+
+test {
+    dependsOn taskBuildDockerImage
+    finalizedBy taskRemoveDockerImage
+}

JShellAPI/src/main/java/org/togetherjava/jshellapi/rest/ApiEndpoints.java

@@ -0,0 +1,17 @@
+package org.togetherjava.jshellapi.rest;
+
+/**
+ * This class holds endpoints mentioned in controllers. The main objective is to keep endpoints
+ * synchronized with testing classes.
+ *
+ * @author Firas Regaieg
+ */
+public final class ApiEndpoints {
+    private ApiEndpoints() {}
+
+    public static final String BASE = "/jshell";
+    public static final String EVALUATE = "/eval";
+    public static final String SINGLE_EVALUATE = "/single-eval";
+    public static final String SNIPPETS = "/snippets";
+    public static final String STARTING_SCRIPT = "/startup_script";
+}

JShellAPI/src/main/java/org/togetherjava/jshellapi/rest/JShellController.java

@@ -15,13 +15,13 @@
 
 import java.util.List;
 
-@RequestMapping("jshell")
+@RequestMapping(ApiEndpoints.BASE)
 @RestController
 public class JShellController {
     private JShellSessionService service;
     private StartupScriptsService startupScriptsService;
 
-    @PostMapping("/eval/{id}")
+    @PostMapping(ApiEndpoints.EVALUATE + "/{id}")
     public JShellResult eval(@PathVariable String id,
             @RequestParam(required = false) StartupScriptId startupScriptId,
             @RequestBody String code) throws DockerException {
@@ -32,7 +32,7 @@ public JShellResult eval(@PathVariable String id,
                     "An operation is already running"));
     }
 
-    @PostMapping("/eval")
+    @PostMapping(ApiEndpoints.EVALUATE)
     public JShellResultWithId eval(@RequestParam(required = false) StartupScriptId startupScriptId,
             @RequestBody String code) throws DockerException {
         JShellService jShellService = service.session(startupScriptId);
@@ -42,7 +42,7 @@ public JShellResultWithId eval(@RequestParam(required = false) StartupScriptId s
                             "An operation is already running")));
     }
 
-    @PostMapping("/single-eval")
+    @PostMapping(ApiEndpoints.SINGLE_EVALUATE)
     public JShellResult singleEval(@RequestParam(required = false) StartupScriptId startupScriptId,
             @RequestBody String code) throws DockerException {
         JShellService jShellService = service.oneTimeSession(startupScriptId);
@@ -51,7 +51,7 @@ public JShellResult singleEval(@RequestParam(required = false) StartupScriptId s
                     "An operation is already running"));
     }
 
-    @GetMapping("/snippets/{id}")
+    @GetMapping(ApiEndpoints.SNIPPETS + "/{id}")
     public List<String> snippets(@PathVariable String id,
             @RequestParam(required = false) boolean includeStartupScript) throws DockerException {
         validateId(id);
@@ -71,7 +71,7 @@ public void delete(@PathVariable String id) throws DockerException {
         service.deleteSession(id);
     }
 
-    @GetMapping("/startup_script/{id}")
+    @GetMapping(ApiEndpoints.STARTING_SCRIPT + "/{id}")
     public String startupScript(@PathVariable StartupScriptId id) {
         return startupScriptsService.get(id);
     }

JShellAPI/src/main/java/org/togetherjava/jshellapi/service/DockerService.java

@@ -7,9 +7,11 @@
 import com.github.dockerjava.core.DefaultDockerClientConfig;
 import com.github.dockerjava.core.DockerClientImpl;
 import com.github.dockerjava.httpclient5.ApacheDockerHttpClient;
+import jakarta.el.PropertyNotFoundException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.DisposableBean;
+import org.springframework.beans.factory.annotation.Value;
 import org.springframework.lang.Nullable;
 import org.springframework.stereotype.Service;
 
@@ -29,6 +31,9 @@ public class DockerService implements DisposableBean {
 
     private final DockerClient client;
 
+    @Value("${jshell-wrapper.image-name}")
+    private String jshellWrapperImageName;
+
     public DockerService(Config config) {
         DefaultDockerClientConfig clientConfig =
                 DefaultDockerClientConfig.createDefaultConfigBuilder().build();
@@ -59,22 +64,33 @@ private void cleanupLeftovers(UUID currentId) {
 
     public String spawnContainer(long maxMemoryMegs, long cpus, @Nullable String cpuSetCpus,
             String name, Duration evalTimeout, long sysoutLimit) throws InterruptedException {
-        String imageName = "togetherjava.org:5001/togetherjava/jshellwrapper";
+        String imageName = Optional.ofNullable(this.jshellWrapperImageName)
+            .orElseThrow(() -> new PropertyNotFoundException(
+                    "unable to find jshellWrapper image name property"));
+
+        String[] imageNameParts = imageName.split(":master");
+
+        if (imageNameParts.length != 1) {
+            throw new IllegalArgumentException("invalid jshellWrapper image name");
+        }
+
+        String baseImageName = imageNameParts[0];
+
         boolean presentLocally = client.listImagesCmd()
-            .withFilter("reference", List.of(imageName))
+            .withFilter("reference", List.of(baseImageName))
             .exec()
             .stream()
             .flatMap(it -> Arrays.stream(it.getRepoTags()))
             .anyMatch(it -> it.endsWith(":master"));
 
         if (!presentLocally) {
-            client.pullImageCmd(imageName)
+            client.pullImageCmd(baseImageName)
                 .withTag("master")
                 .exec(new PullImageResultCallback())
                 .awaitCompletion(5, TimeUnit.MINUTES);
         }
 
-        return client.createContainerCmd(imageName + ":master")
+        return client.createContainerCmd(baseImageName + ":master")
             .withHostConfig(HostConfig.newHostConfig()
                 .withAutoRemove(true)
                 .withInit(true)

JShellAPI/src/main/resources/META-INF/additional-spring-configuration-metadata.json

@@ -0,0 +1,8 @@
+{
+  "properties": [
+    {
+      "name": "jshell-wrapper.image-name",
+      "type": "java.lang.String",
+      "description": "JShellWrapper image name injected from the top-level gradle build file."
+  }
+] }

JShellAPI/src/main/resources/application.yaml

@@ -20,6 +20,9 @@ jshellapi:
   dockerResponseTimeout: 60
   dockerConnectionTimeout: 60
 
+jshell-wrapper:
+  image-name: ${jShellWrapperImageName}
+
 server:
   error:
     include-message: always

JShellAPI/src/test/java/org/togetherjava/jshellapi/JShellApiTests.java

@@ -0,0 +1,90 @@
+package org.togetherjava.jshellapi;
+
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.Test;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.boot.test.context.SpringBootTest;
+import org.springframework.test.context.ContextConfiguration;
+import org.springframework.test.web.reactive.server.WebTestClient;
+
+import org.togetherjava.jshellapi.dto.JShellResult;
+import org.togetherjava.jshellapi.dto.JShellSnippetResult;
+import org.togetherjava.jshellapi.dto.SnippetStatus;
+import org.togetherjava.jshellapi.dto.SnippetType;
+import org.togetherjava.jshellapi.rest.ApiEndpoints;
+
+import java.time.Duration;
+import java.util.List;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * This class holds integration tests for JShellAPI. It depends on gradle building image task, fore
+ * more information check "test" section in gradle.build file.
+ *
+ * @author Firas Regaieg
+ */
+@ContextConfiguration(classes = Main.class)
+@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
+public class JShellApiTests {
+
+    @Autowired
+    private WebTestClient webTestClient;
+
+    @Test
+    @DisplayName("When posting code snippet, evaluate it then returns successfully result")
+    public void evaluateCodeSnippetTest() {
+
+        final String testEvalId = "test";
+
+        // -- performing a first code snippet execution
+
+        final String firstCodeExpression = "int a = 2+2;";
+
+        final JShellSnippetResult firstCodeSnippet = new JShellSnippetResult(SnippetStatus.VALID,
+                SnippetType.ADDITION, 1, firstCodeExpression, "4");
+        final JShellResult firstCodeExpectedResult =
+                getJShellResultDefaultInstance(firstCodeSnippet);
+
+        assertThat(testEval(testEvalId, firstCodeExpression)).isEqualTo(firstCodeExpectedResult);
+
+        // -- performing a second code snippet execution
+
+        final String secondCodeExpression = "a * 2";
+
+        final JShellSnippetResult secondCodeSnippet = new JShellSnippetResult(SnippetStatus.VALID,
+                SnippetType.ADDITION, 2, secondCodeExpression, "8");
+
+        final JShellResult secondCodeExpectedResult =
+                getJShellResultDefaultInstance(secondCodeSnippet);
+
+        assertThat(testEval(testEvalId, secondCodeExpression)).isEqualTo(secondCodeExpectedResult);
+    }
+
+    private JShellResult testEval(String testEvalId, String codeInput) {
+        final String endpoint =
+                String.join("/", ApiEndpoints.BASE, ApiEndpoints.EVALUATE, testEvalId);
+
+        JShellResult result = this.webTestClient.mutate()
+            .responseTimeout(Duration.ofSeconds(6))
+            .build()
+            .post()
+            .uri(endpoint)
+            .bodyValue(codeInput)
+            .exchange()
+            .expectStatus()
+            .isOk()
+            .expectBody(JShellResult.class)
+            .value((JShellResult evalResult) -> assertThat(evalResult).isNotNull())
+            .returnResult()
+            .getResponseBody();
+
+        assertThat(result).isNotNull();
+
+        return result;
+    }
+
+    private static JShellResult getJShellResultDefaultInstance(JShellSnippetResult snippetResult) {
+        return new JShellResult(List.of(snippetResult), null, false, "");
+    }
+}