Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update gatling #3

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions rungatling.sh
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# Download gatling bundle from here, extract it
# https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/3.4.1/gatling-charts-highcharts-bundle-3.4.1-bundle.zip
GATLING=$DIR/gatling-charts-highcharts-bundle-3.4.1/bin/gatling.sh
# https://repo1.maven.org/maven2/io/gatling/highcharts/gatling-charts-highcharts-bundle/3.8.4/gatling-charts-highcharts-bundle-3.8.4-bundle.zip
GATLING=$DIR/gatling-charts-highcharts-bundle-3.8.4/bin/gatling.sh

URL=${1:-http://localhost/domjudge}
echo "Running gatling against: $URL"

export JAVA_OPTS="-Dbaseurl=$URL"
$GATLING --results-folder $PWD/reports --simulations-folder $PWD/simulations --resources-folder $PWD/bodies --simulation domjudge.ContestSimulation
$GATLING --run-mode local --results-folder $PWD/reports --simulations-folder $PWD/simulations --resources-folder $PWD/bodies --run-description "domjudge" --simulation domjudge.ContestSimulation
303 changes: 303 additions & 0 deletions seeding/accounts.json

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions seeding/groups.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[
{"id":"4242","name":"Gatling Accounts"}
]
3 changes: 3 additions & 0 deletions seeding/organizations.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[
{"id":"4242","name":"Gatling Organization","formal_name":"Gatling Organization to associate"}
]
31 changes: 31 additions & 0 deletions seeding/seed.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
USERNAME=admin
PASSWORD=real_secret_password
DJ_URL=http://localhost/domjudges

# Set data source to "config data external" so that importing things works properly
# http -a "$USERNAME":"$PASSWORD" -f PUT $DJ_URL/api/v4/config data_source=1

# Import a gatling organization(affiliation) and group(category)
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/users/organizations json@seeding/organizations.json
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/users/groups json@seeding/groups.json

# Load some accounts/teams
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/users/accounts json@seeding/accounts.json
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/users/teams json@seeding/teams.json


# The accounts/teams just need to be part of some contest, with a hello world problem. The specific name/etc is not important.
# Create the gatling contest, starting now and running for 9999 hours (It will be public, since there's no exposed api to make it private :sadface:)
cat > seeding/contest.json <<EOF
{
"name": "gatling",
"short-name": "gatling",
"start-time": "$(date -Iseconds)",
"duration": "9999:00:00.000",
"penalty-time": 20
}
EOF
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/contests json@seeding/contest.json

# Add a problem to the contest
http -a "$USERNAME":"$PASSWORD" -f POST $DJ_URL/api/v4/contests/gatling/problems zip@bodies/hello-testcase.zip
252 changes: 252 additions & 0 deletions seeding/teams.json

Large diffs are not rendered by default.

192 changes: 192 additions & 0 deletions simulations/ContestSimulation.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
package domjudge;

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;
import java.time.Duration;
import java.io.*;
import java.util.*;
import java.util.stream.Stream;
import java.util.function.*;
import java.util.concurrent.atomic.AtomicInteger;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;


public class ContestSimulation extends Simulation {
// A fixed password (see seeding/accounts.json)
String PASSWORD="secret";

// How many "actions" a team will take (actions are things like submitting, requesting clarification, viewing a scoreboard)
int NUM_TEAM_ACTIONS = 5;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adjust this to make a team "user" last longer/do more things.


// Maximum amount of time the scenario can run
int MAX_MINUTES = 5;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sets a cap for how long this will run (which also sets how long a spectator will keep refreshing for.


private ChainBuilder login() {
return exec(
http("Login page get csrf")
.get("/login")
.check(
css("input[name='_csrf_token']", "value")
.find()
.saveAs("csrftoken")
))
.exec(http("Login Request")
.post("/login")
.formParam("_username", "#{user}")
.formParam("_password", PASSWORD)
.formParam("_csrf_token", "#{csrftoken}")
);
}

private ChainBuilder submit(String langid, String filename) {
return exec(
http("Get submit form csrf from team page")
.get("/team/submit")
.check(
css("input[id='submit_problem__token']", "value").find()
.saveAs("csrftoken"),
regex("<option value=\"([^\"]*)\">hello").find()
.saveAs("problem_id")
)
).exec(
http(String.format("Submit solution %s", langid))
.post("/team/submit")
.formParam("submit_problem[_token]","#{csrftoken}")
.formParam("submit_problem[problem]","#{problem_id}")
.formParam("submit_problem[language]", langid)
.formUpload("submit_problem[code][]", filename)
.formParam("submit", "")
);
}

private ChainBuilder requestClarification() {
return exec(
http("Get request clarification form")
.get("/team/clarifications/add")
.check(
css("input[id=team_clarification__token", "value").find()
.saveAs("csrftoken"),
// get and save the clarification subject id for the "General issue" category
regex("<option value=\"([^\"]*-general)\">General").find()
.saveAs("clarification_subject")
)
).exec(
http("Request Clarification")
.post("/team/clarifications/add")
.formParam("team_clarification[recipient]", "dummy")
.formParam("team_clarification[subject]", "#{clarification_subject}")
.formParam("team_clarification[message]", "#{user} needs help")
.formParam("team_clarification[_token]", "#{csrftoken}")
.formParam("submit", "")
);
}

/*********************** Spectator Scenario ****************************************************************
* Scenario that just grabs the public scoreboard every 30 seconds. The scoreboard page autorefreshes
* every 30s, so this is a good approximation of their activity.
*/
HttpRequestActionBuilder publicScoreboard = http("Public Scoreboard").get("/public");
ChainBuilder monitorScoreboard = exec(poll().every(30).exec(publicScoreboard)).pause(Duration.ofMinutes(MAX_MINUTES));
ScenarioBuilder spectatorScenario = scenario("SpectatorScenario").exec(monitorScoreboard);


/*********************** Team Scenario *********************************************************************
* This pretends to be a team, it polls the team page every 30s and randomly performs various actions
*/
// Feeder for team accounts
AtomicInteger counter = new AtomicInteger(1);
Iterator<Map<String,Object>> feeder =
Stream.generate((Supplier<Map<String,Object>>) () -> {
String username = String.format("gatling%04d", counter.getAndIncrement());
return Collections.singletonMap("user", username);
}
).iterator();

HttpRequestActionBuilder teamPage = http("Team Page").get("/team");
ChainBuilder teamSubmitC = submit("c", "test-hello.c");
ChainBuilder teamSubmitCpp = submit("cpp", "test-hello.c++");
ChainBuilder teamSubmitJava = submit("java", "test-hello.java");
ChainBuilder teamSubmitKotlin = submit("kt", "test-hello.kt");
ChainBuilder teamSubmitPython = submit("py3", "test-hello.py3");
ChainBuilder teamRequestClarification = requestClarification();
ChainBuilder teamProblems = exec(http("Team Problemset").get("/team/problems"));
ChainBuilder teamScoreboard = exec(http("Team Scoreboard").get("/team/scoreboard"));
ScenarioBuilder teamScenario = scenario("TeamScenario")
.feed(feeder)
.exec(http("Homepage").get("/"))
.exec(publicScoreboard) // Fetch the public scoreboard
.pause(1,5) // Brief delay
.exec(login()) // Log in as the team

// Mimic the behavior of the team page, it autorefreshes every 30s
.exec(
poll().pollerName("teampage")
.every(30)
.exec(teamPage)
)

// While we're polling the teampage, lets pretend to do some actions like a real team might
.repeat(NUM_TEAM_ACTIONS, "i").on(
exec(
randomSwitch().on(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the random actions that a team will do, and the probabilities of each one.

// 20% chance of making a submission
Choice.withWeight(20.0, exec(
uniformRandomSwitch().on(
exec(teamSubmitC),
exec(teamSubmitCpp),
exec(teamSubmitJava),
exec(teamSubmitKotlin),
exec(teamSubmitPython)
)
)),
// 10% of requesting clarification
Choice.withWeight(10.0, teamRequestClarification),
// 30% of loading problems page
Choice.withWeight(30.0, teamProblems),
// 40% of loading scoreboard
Choice.withWeight(40.0, teamScoreboard)
)
)
// Sleep anywhere between 5s and 1 minute before doing another action
.pause(5,60)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much time between each team "action". This is in seconds, so teams will take an action between 5 and 60s after their last action.

)

// Stop polling the teampage
.exec(
poll().pollerName("teampage").stop()
)

;


HttpProtocolBuilder httpProtocol = http
.baseUrl(System.getProperty("baseurl"))
.inferHtmlResources() // load all dependent things like images/js/css/etc
.nameInferredHtmlResourcesAfterPath()
;

{
setUp(
// teamScenario.injectOpen(atOnceUsers(5)),
teamScenario.injectClosed(
incrementConcurrentUsers(10) // Batches of this many users
.times(10) // how many "levels" to test
.eachLevelLasting(60) // hold the number of users for this duration
.separatedByRampsLasting(60) // Take 60s between adding each user
.startingFrom(10) // Start with 10 users
),
// spectatorScenario.injectOpen(atOnceUsers(10))
spectatorScenario.injectClosed(
incrementConcurrentUsers(10) // Batches of this many spectators
.times(10) // how many "levels" to test
.eachLevelLasting(60) // hold the number of users for this duration
.separatedByRampsLasting(60) // Take 60s between adding each user
.startingFrom(10) // Start with 10 users
)
Comment on lines +173 to +187
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the bit you'll want to adjust to tune how many concurrent users are hammering the system. Probably play with these things until it falls over.

)
.maxDuration(Duration.ofMinutes(MAX_MINUTES))
.protocols(httpProtocol);
}
}
35 changes: 0 additions & 35 deletions simulations/domjudge/AdminSetup.scala

This file was deleted.

Loading