Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,5 @@ project/plugins/project
project/sbt-ui.sbt
target/
credentials.sbt

*.csv
40 changes: 15 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,17 @@

[![image](https://upload.wikimedia.org/wikipedia/en/a/a7/MassiveAttackBlueLines.jpg)](https://en.wikipedia.org/wiki/Blue_Lines)

`massive-attack` is a simple and configurable load generator test tool written in Scala originally to test Apache Thrift endpoints, but it can be
used to benchmark any method that returns a Scala or Twitter `Future`.

<h2>Why?</h2>
Proprietor: RMS

Because I needed to load test a Thrift endpoint in one of my APIs, and could not find an easy to use tool after looking around for days, at least
not one that did not require me to develop JMeter plugins, or one that has been updated in the past few years.

<h2>How?</h2>

So I decided to create one which is easy to use:

1. It can easily be added as a dependency to your API or application
2. It is configurable as much or as little as you need it to be
3. It is extensible
Load generator test tool written in Scala used to benchmark any method that returns a Scala or Twitter `Future`.
Project forked from [massive-attack](https://github.com/delprks/massive-attack) project created and mantained by [Daniel Parks](https://github.com/delprks)

<h2>Usage</h2>
## Usage

1. Add it as a dependency to `build.sbt`:

`libraryDependencies ++= Seq("com.delprks" %% "massive-attack" % "1.0.0" % "test")`
`libraryDependencies += "bbc.rms" %% "massive-attack" % <release> % "test")`

2. Create your test in [ScalaTest](http://www.scalatest.org) or [Specs2](https://etorreborre.github.io/specs2) (this library might change to be a testing framework in future)

Expand Down Expand Up @@ -58,47 +48,47 @@ And (if enabled) generate a report containing the test results:

![image](https://user-images.githubusercontent.com/8627976/41814268-92468eac-773f-11e8-8076-88b4ef9e17e1.png)

<h2>Test properties</h2>
## Test properties

Following properties are available and configurable through `MassiveAttackProperties`:

<h3>invocations</h3>
### invocations

Specifies how many times the method should be invoked.

<h3>threads</h3>
### threads

You can set how many threads you want to run the load test on - beware that Thrift clients can run only on single threads.

<h3>duration</h3>
### duration

Specifies how long the method should be tested for in seconds - whichever comes first (duration or invocations) determines the length of the test.

<h3>warmUp</h3>
### warmUp

This is by default set to `true` to avoid cold start times affecting the test results - set it to false if you want to test cold starts.

<h3>warmUpInvocations</h3>
### warmUpInvocations

If `warmUp` is set to true, `warmUpInvocations` determines how many times the method should be invoked before the load test starts.

<h3>spikeFactor</h3>
### spikeFactor

This is used to decide which response times should be considered as spikes, by multiplying the average response time and `spikeFactor`. It has the default
value of `3.0`.

<h3>verbose</h3>
### verbose

Set this to true if you want to see invocation times when the load test is in progress.

<h3>report</h3>

Set this to true if you want to save test results in a CSV file.

<h3>reportName</h3>
### reportName

If `report` is set to true, results will be saved to this file. If no `reportName` is specified, one will be generated.

# License

`massive-attack` is open source software released under the [Apache 2 License](http://www.apache.org/licenses/LICENSE-2.0).
original [massive-attack](https://github.com/delprks/massive-attack) is an open source software released under the [Apache 2 License](http://www.apache.org/licenses/LICENSE-2.0).
126 changes: 65 additions & 61 deletions build.sbt
Original file line number Diff line number Diff line change
@@ -1,74 +1,78 @@
name := "massive-attack"
import sbt.Keys.pomExtra
import sbt.{Classpaths, Test}

crossScalaVersions := Seq("2.12.6", "2.11.12")

organization := "com.delprks"
ThisBuild / organization := "bbc.rms"
ThisBuild / scalaVersion := "2.13.2"
ThisBuild / name := "massive-attack"

scalacOptions := Seq("-unchecked", "-deprecation", "-feature", "-encoding", "utf8")

javacOptions ++= Seq("-source", "1.8", "-target", "1.8")

publishTo := {
val isSnapshotValue = isSnapshot.value
val nexus = "https://oss.sonatype.org/"
if (isSnapshotValue) Some("snapshots" at nexus + "content/repositories/snapshots")
else Some("releases" at nexus + "service/local/staging/deploy/maven2")
}

publishMavenStyle := true

publishArtifact in Test := false

parallelExecution in Test := false

releaseCrossBuild := true

sbtrelease.ReleasePlugin.autoImport.releasePublishArtifactsAction := PgpKeys.publishSigned.value

sbtrelease.ReleasePlugin.autoImport.releaseCrossBuild := false

SbtPgp.autoImport.useGpg := true
resolvers ++= Seq("BBC Artifactory" at "https://artifactory.dev.bbc.co.uk/artifactory/repo/") :+ Classpaths.typesafeReleases

lazy val mavenStyleSettings = Seq(
publishMavenStyle := true,
pomIncludeRepository := {
_ => false
},
pomExtra := {
<url>https://github.com/delprks/massive-attack</url>
<licenses>
<license>
<name>Apache 2</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<url>git@github.com:delprks/massive-attack.git</url>
<connection>scm:git@github.com:delprks/massive-attack.git</connection>
</scm>
<developers>
<developer>
<id>delprks</id>
<name>Daniel Parks</name>
<url>http://github.com/delprks</url>
</developer>
</developers>
}
)

SbtPgp.autoImport.useGpgAgent := true

libraryDependencies ++= Seq(
"com.twitter" %% "util-core" % "18.5.0",
"com.typesafe.akka" %% "akka-actor" % "2.5.13",
"com.typesafe.akka" %% "akka-testkit" % "2.5.13" % Test,
"org.scalatest" %% "scalatest" % "3.0.5" % Test
lazy val publishSettings = Seq(
version := scala.util.Properties.envOrElse("BUILD_VERSION", "0.1-SNAPSHOT"),
publishArtifact in (Test, packageBin) := true,
publishMavenStyle := true,
publishTo := Some("BBC Repository" at "https://artifactory.dev.bbc.co.uk/artifactory/int-bbc-releases"),
credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
)

releaseTagComment := s"Releasing ${(version in ThisBuild).value}"
releaseCommitMessage := s"= Setting version to ${(version in ThisBuild).value}"

pomIncludeRepository := {
_ => false
}

pomExtra := {
<url>https://github.com/delprks/massive-attack</url>
<licenses>
<license>
<name>Apache 2</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<url>git@github.com:delprks/massive-attack.git</url>
<connection>scm:git@github.com:delprks/massive-attack.git</connection>
</scm>
<developers>
<developer>
<id>delprks</id>
<name>Daniel Parks</name>
<url>http://github.com/delprks</url>
</developer>
</developers>
}

connectInput in run := true
lazy val testSettings = Seq(
Test / publishArtifact := false,
Test / parallelExecution := false
)

fork in run := true

lazy val root = project in file(".")
libraryDependencies ++= Seq(
"com.twitter" %% "util-core" % "20.5.0",
"com.typesafe.akka" %% "akka-actor" % "2.6.5",
"com.typesafe.akka" %% "akka-testkit" % "2.6.5" % Test,
"org.scalatest" %% "scalatest" % "3.1.2" % Test
) ++ (CrossVersion.partialVersion(scalaVersion.value) match {
case Some((2, major)) if major <= 12 =>
Seq()
case _ =>
Seq("org.scala-lang.modules" %% "scala-parallel-collections" % "0.2.0")
})


run /connectInput := true

run / fork := true

lazy val `massive-attack` = (project in file("."))
.settings(publishSettings)
.settings(testSettings)
.settings(mavenStyleSettings)
2 changes: 1 addition & 1 deletion project/build.properties
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sbt.version = 1.1.6
sbt.version = 1.3.10
3 changes: 1 addition & 2 deletions project/plugins.sbt
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
resolvers += Classpaths.sbtPluginReleases

addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.1.0")
addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.7")
addSbtPlugin("com.jsuereth" % "sbt-pgp" % "2.0.0")
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.delprks.massiveattack
package bbc.rms.massiveattack

import com.twitter.util.{Future => TwitterFuture}

Expand Down
3 changes: 3 additions & 0 deletions src/main/scala/bbc/rms/massiveattack/MassiveAttackProps.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
package bbc.rms.massiveattack

trait MassiveAttackProps
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.delprks.massiveattack
package bbc.rms.massiveattack

trait MassiveAttackResult {
def toString: String
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
package com.delprks.massiveattack.method
package bbc.rms.massiveattack.method

import java.util.concurrent.Executors

import akka.util.Timeout
import com.delprks.massiveattack.MassiveAttack
import com.delprks.massiveattack.method.result.{MethodDurationResult, MethodPerformanceResult}
import com.delprks.massiveattack.method.util.{MethodOps, ResultOps}
import bbc.rms.massiveattack.MassiveAttack
import bbc.rms.massiveattack.method.result.{MethodDurationResult, MethodPerformanceResult}
import bbc.rms.massiveattack.method.util.{MethodOps, ResultOps}
import scala.collection.parallel.CollectionConverters._


import scala.collection.mutable.ListBuffer
import scala.collection.parallel.ForkJoinTaskSupport
import scala.collection.parallel.mutable.ParArray
import com.twitter.util.{Future => TwitterFuture}

import scala.collection.parallel.ForkJoinTaskSupport
import scala.collection.parallel.mutable.ParArray
import scala.concurrent.duration._
import scala.concurrent.forkjoin.ForkJoinPool
import scala.concurrent.{ExecutionContext, Future => ScalaFuture}
import scala.reflect.ClassTag

Expand All @@ -37,8 +38,8 @@ class MethodPerformance(props: MethodPerformanceProps = MethodPerformanceProps()

val testStartTime = System.currentTimeMillis()
val testEndTime = testStartTime + props.duration * 1000
val parallelInvocation: ParArray[Int] = (1 to props.invocations).toParArray
val forkJoinPool = new ForkJoinPool(props.threads)
val parallelInvocation: ParArray[Int] = (1 to props.invocations).toArray.par
val forkJoinPool = new java.util.concurrent.ForkJoinPool(props.threads)

parallelInvocation.tasksupport = new ForkJoinTaskSupport(forkJoinPool)

Expand All @@ -62,10 +63,11 @@ class MethodPerformance(props: MethodPerformanceProps = MethodPerformanceProps()

println(Console.RED + s"Invoking method ${props.invocations} times - or maximum ${props.duration} seconds - on ${props.threads} threads" + Console.RESET)


val testStartTime = System.currentTimeMillis()
val testEndTime = testStartTime + props.duration * 1000
val parallelInvocation: ParArray[Int] = (1 to props.invocations).toParArray
val forkJoinPool = new scala.concurrent.forkjoin.ForkJoinPool(props.threads)
val parallelInvocation: ParArray[Int] = (1 to props.invocations).toArray.par
val forkJoinPool = new java.util.concurrent.ForkJoinPool(props.threads)

parallelInvocation.tasksupport = new ForkJoinTaskSupport(forkJoinPool)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.delprks.massiveattack.method
package bbc.rms.massiveattack.method

import com.delprks.massiveattack.MassiveAttackProps
import bbc.rms.massiveattack.MassiveAttackProps

case class MethodPerformanceProps(
invocations: Int = 1000,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
package com.delprks.massiveattack.method.result
package bbc.rms.massiveattack.method.result

case class MethodDurationResult(duration: Long, endTime: Long)
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
package com.delprks.massiveattack.method.result
package bbc.rms.massiveattack.method.result

import com.delprks.massiveattack.MassiveAttackResult
import bbc.rms.massiveattack.MassiveAttackResult

case class MethodPerformanceResult(
responseTimeMin: Int,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package com.delprks.massiveattack.method.result
package bbc.rms.massiveattack.method.result

import akka.actor.Actor

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package com.delprks.massiveattack.method.util
package bbc.rms.massiveattack.method.util

import akka.actor.{ActorSystem, Props}
import com.delprks.massiveattack.method.result.{GetStats, MethodDurationResult, MethodStatsRecorder}
import com.twitter.util.{Future => TwitterFuture}

import scala.collection.mutable.ListBuffer
Expand All @@ -11,6 +10,7 @@ import scala.reflect.ClassTag
import akka.actor._
import akka.pattern.ask
import akka.util.Timeout
import bbc.rms.massiveattack.method.result.{GetStats, MethodDurationResult, MethodStatsRecorder}

import scala.concurrent.{ExecutionContext, Future}
import scala.concurrent.duration._
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
package com.delprks.massiveattack.method.util
package bbc.rms.massiveattack.method.util

import java.nio.charset.StandardCharsets
import java.nio.file.{Files, Paths}
import java.text.SimpleDateFormat
import java.util.Calendar

import com.delprks.massiveattack.method.MethodPerformanceProps
import com.delprks.massiveattack.method.result.{MethodDurationResult, MethodPerformanceResult}
import bbc.rms.massiveattack.method.MethodPerformanceProps
import bbc.rms.massiveattack.method.result.{MethodDurationResult, MethodPerformanceResult}

import scala.concurrent.{ExecutionContext, Future}
import scala.collection.mutable.ListBuffer
Expand All @@ -15,7 +15,8 @@ class ResultOps()(implicit ec: ExecutionContext) {

def testResults(results: Future[ListBuffer[MethodDurationResult]], testProps: MethodPerformanceProps): Future[MethodPerformanceResult] = results map { response =>
val responseDuration = response.map(_.duration.toInt)
val average = avg(responseDuration)
val responseDurationSeq = responseDuration.toSeq
val average = avg(responseDurationSeq)
val invocationSeconds = response.map(_.endTime / 1000)
val requestTimesPerSecond = invocationSeconds.groupBy(identity).map(_._2.size)
val spikeBoundary = (average * testProps.spikeFactor).toInt
Expand All @@ -27,8 +28,8 @@ class ResultOps()(implicit ec: ExecutionContext) {
val testResult = MethodPerformanceResult(
responseTimeMin = responseDuration.min,
responseTimeMax = responseDuration.max,
responseTime95tile = percentile(95)(responseDuration),
responseTime99tile = percentile(99)(responseDuration),
responseTime95tile = percentile(95)(responseDurationSeq),
responseTime99tile = percentile(99)(responseDurationSeq),
responseTimeAvg = average,
rpsMin = requestTimesPerSecond.min,
rpsMax = requestTimesPerSecond.max,
Expand Down
Loading