Como posso usar a base de testes de faísca com o Maven?

Gostava de descobrir como testar o mesmo código de ignição. Pesquisei no Google e encontrei a base de faísca. Bem, agora eu gostaria de experimentá-lo, mas eu não sou capaz de executá-lo com Maven.

primeiro, Tenho o código scala que é embalável com mvn package e tem a seguinte estrutura do projecto:

pom.xml
src/main/scala/com/test/spark/mycode.scala
src/test/scala/com/test/spark/test.scala

o problema é que quando corro mvn test não faz o teste em src/main/com/test/spark/test.scala. O ensaio é, na verdade, o primeiro exemplo da base de ensaio de faíscas wiki . A saída de maven é:

[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ mycode ---
[INFO] Nothing to compile - all classes are up to date

e o meu pom.xml parece-se com isto:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.test.spark</groupId>
  <artifactId>mycode</artifactId>
  <version>0.0.1</version>
  <name>${project.artifactId}</name>
  <description>Simple wtest</description>
  <inceptionYear>2017</inceptionYear>

  <!-- change from 1.6 to 1.7 depending on Java version -->
  <properties>
    <maven.compiler.source>1.6</maven.compiler.source>
    <maven.compiler.target>1.6</maven.compiler.target>
    <encoding>UTF-8</encoding>
    <scala.version>2.11.5</scala.version>
    <scala.compat.version>2.11</scala.compat.version>
    <spark.version>1.6.1</spark.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.scala-lang</groupId>
      <artifactId>scala-library</artifactId>
      <version>${scala.version}</version>
    </dependency>

    <!-- Spark dependency -->
    <dependency>         
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- Spark sql dependency -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- Spark hive dependency -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-hive_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- spark-testing-base dependency -->
    <dependency>
        <groupId>com.holdenkarau</groupId>
        <artifactId>spark-testing-base_2.11</artifactId>
        <version>${spark.version}_0.6.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.scalactic</groupId>
        <artifactId>scalactic_2.11</artifactId>
        <version>3.0.1</version>
    </dependency>
    <dependency>
        <groupId>org.scalatest</groupId>
        <artifactId>scalatest_2.11</artifactId>
        <version>3.2.0-SNAP5</version>
        <scope>test</scope>
    </dependency>
 </dependencies>

  <build>
    <sourceDirectory>src/main/scala</sourceDirectory>
    <testSourceDirectory>src/test/scala</testSourceDirectory>
    <!-- Create JAR with all dependencies -->
    <plugins>
       <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
        </configuration>
      </plugin>
      <plugin>
        <!-- see http://davidb.github.com/scala-maven-plugin -->
        <groupId>net.alchim31.maven</groupId>
        <artifactId>scala-maven-plugin</artifactId>
        <version>3.2.2</version>
        <configuration>
            <scalaVersion>${scala.version}</scalaVersion>
            <scalaCompatVersion>${scala.compat.version}</scalaCompatVersion>
        </configuration>
        <executions>
          <execution>
            <phase>compile</phase>
            <goals>
              <goal>compile</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <!-- for testing scala code -->
      <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.20</version>
          <configuration>
            <argLine>-Xmx2048m -XX:MaxPermSize=2048m</argLine>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>
Qualquer ajuda agradecida. Obrigado!

Editar 1:

adicionei isto como test.scala para chumbar no teste:

class test extends FunSuite with SharedSparkContext {
  test("test initializing spark context") {
    val list = List(1, 2, 3, 4)
    val rdd = sc.parallelize(list)
    val list2 = List(1, 2, 3, 4, 5)

    assert(rdd.count === list2.length)
  }
}
Mas ainda tenho ...
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ mycode ---
[INFO] Nothing to compile - all classes are up to date
[INFO]

[INFO] --- maven-surefire-plugin:2.20:test (default-test) @ mycode ---
[INFO]------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]------------------------------------------------------------------------

embora a comparação em assert seja falsa como se o teste não funcionasse. :(

Editar 2: Eu mudei o pom para com este plugin, ainda sem alterações:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.20</version>
  <configuration>
    <includes>
      <include>**/*Test.scala</include>
    </includes>
    <argLine>-Xmx2048m -XX:MaxPermSize=2048m</argLine>
  </configuration>
</plugin>       

editar 3: Adicionado <goal>testCompile</goal> a net.alchim31.maven para compilar a classe de teste. Embora o teste ainda não funcione. O que falta para fazer o teste??

editar 4: Obrigado pelos seus comentários. Mudei o pom para o seguinte:

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

<groupId>com.test.spark</groupId>
<artifactId>testJavaAndScala</artifactId>
<version>1.0-SNAPSHOT</version>
<name>Test for Java + Scala compilation</name>
<description>Test for Java + Scala compilation</description>
<inceptionYear>2017</inceptionYear>

<properties>
    <scala.version>2.11.5</scala.version>
    <scala.compat.version>2.11</scala.compat.version>
    <spark.version>1.6.1</spark.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
        <version>${scala.version}</version>
    </dependency>
    <!-- Spark dependency -->
    <dependency>         
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- Spark sql dependency -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- Spark hive dependency -->
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-hive_${scala.compat.version}</artifactId>
        <version>${spark.version}</version>
    </dependency>
    <!-- Spark-testing-base -->
    <dependency>
        <groupId>com.holdenkarau</groupId>
        <artifactId>spark-testing-base_${scala.compat.version}</artifactId>
        <version>${spark.version}_0.6.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <pluginManagement>
        <plugins>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.2.1</version>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.0.2</version>
            </plugin>
        </plugins>
    </pluginManagement>
    <plugins>
        <plugin>
            <groupId>net.alchim31.maven</groupId>
            <artifactId>scala-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>scala-compile-first</id>
                    <phase>process-resources</phase>
                    <goals>
                        <goal>add-source</goal>
                        <goal>compile</goal>
                    </goals>
                </execution>
                <execution>
                    <id>scala-test-compile</id>
                    <phase>process-test-resources</phase>
                    <goals>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <executions>
                <execution>
                    <phase>compile</phase>
                    <goals>
                        <goal>compile</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
    </build>

</project>

e esta é a saída mvn test:

[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Test for Java + Scala compilation 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ testJavaAndScala ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/src/main/resources
[INFO]
[INFO] --- scala-maven-plugin:3.2.1:add-source (scala-compile-first) @ testJavaAndScala ---
[INFO] Add Source directory: /home/src/main/scala
[INFO] Add Test Source directory: /home/src/test/scala
[INFO]
[INFO] --- scala-maven-plugin:3.2.1:compile (scala-compile-first) @ testJavaAndScala ---
[INFO] /home/src/main/scala:-1: info: compiling
[INFO] Compiling 1 source files to /home/target/classes at 1497938635103
[INFO] prepare-compile in 0 s
[INFO] compile in 3 s
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:compile (default-compile) @ testJavaAndScala ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:compile (default) @ testJavaAndScala ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ testJavaAndScala ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO]
[INFO] --- scala-maven-plugin:3.2.1:testCompile (scala-test-compile) @ testJavaAndScala ---
[INFO] /home/src/test/scala:-1: info: compiling
[INFO] Compiling 1 source files to /home/target/test-classes at 1497938639022
[INFO] prepare-compile in 0 s
[INFO] compile in 5 s
[INFO]
[INFO] --- maven-compiler-plugin:2.0.2:testCompile (default-testCompile) @ testJavaAndScala ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ testJavaAndScala ---
[INFO] Surefire report directory: /home/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.854 s
[INFO] Finished at: 2017-06-20T08:04:05+02:00
[INFO] Final Memory: 28M/360M
[INFO] ------------------------------------------------------------------------
Posso encontrar a minha classe de teste no directório de destino, mas parece que não foi executada. Porquê? Como eu disse, é apenas o mínimo exmaple e não depende do meu principal codigo.

Author: Ken Jiiii, 2017-06-14

2 answers

O tiro certeiro não encontrará os testes mais escaldantes por si só. Você precisa usar o scalatest-maven-plugin. As instruções estão disponíveis no seu sítio Web Para começar, modifique o seu pom assim.
<!-- disable surefire -- >
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7</version>
  <configuration>
    <skipTests>true</skipTests>
  </configuration>
</plugin>
<!-- enable scalatest -- >
<plugin>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest-maven-plugin</artifactId>
  <version>1.0</version>
  <configuration>
    <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
    <junitxml>.</junitxml>
    <filereports>WDF TestSuite.txt</filereports>
  </configuration>
  <executions>
    <execution>
      <id>test</id>
      <goals>
        <goal>test</goal>
      </goals>
    </execution>
  </executions>
</plugin>
 3
Author: puhlen, 2017-06-15 15:55:11

O teste não pode ser executado devido à estrutura de pastas. A sua estrutura de pastas

src/main/scala/com/test/spark/mycode.scala
src/main/test/com/test/spark/test.scala

A estrutura de pastas deve ser como

src/main/scala/com/test/spark/mycode.scala
src/test/scala/com/test/spark/test.scala

Aqui está a disposição padrão maven.

Espero que isto ajude!

Edição

Falta-lhe um 'plugin' de execução de testes no seu ficheiro pom por favor veja o 'plugin' scala maven

Adicione por favor o seguinte na sua parte de plugin net.alchim31.maven

            <execution>
                <id>scala-test-compile</id>
                <phase>process-test-resources</phase>
                <goals>
                    <goal>testCompile</goal>
                </goals>
            </execution>
 2
Author: Shankar Koirala, 2017-06-15 00:38:59