JMeter

The Apache JMeter™ application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

Apache JMeter may be used to test performance both on static and dynamic resources, Web dynamic applications. It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyze overall performance under different load types.

Additionally, JMeter can help you regression test your application by letting you create test scripts with assertions to validate that your application is returning the results you expect. For maximum flexibility, JMeter lets you create these assertions using regular expressions.

Apache JMeter features include:

• Ability to load and performance test many different applications/server/protocol types:
• Web – HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, …)
• SOAP / REST Webservices
• FTP
• Database via JDBC
• LDAP
• Message-oriented middleware (MOM) via JMS
• Mail – SMTP(S), POP3(S) and IMAP(S)
• Native commands or shell scripts
• TCP
• Java Objects
• Full featured Test IDE that allows fast Test Plan recording (from Browsers or native applications), building and debugging.
• Command-line mode (Non GUI / headless mode) to load test from any Java compatible OS (Linux, Windows, Mac OSX, …)
• A complete and ready to present dynamic HTML report
• Easy correlation through ability to extract data from most popular response formats, HTML, JSON , XML or any textual format
• Complete portability and 100% Java purity.
• Full multi-threading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
• Caching and offline analysis/replaying of test results.
• Highly Extensible core:
• Pluggable Samplers allow unlimited testing capabilities.
• Scriptable Samplers (JSR223-compatible languages like Groovy and BeanShell)
• Several load statistics may be chosen with pluggable timers.
• Data analysis and visualization plugins allow great extensibility as well as personalization.
• Functions can be used to provide dynamic input to a test or provide data manipulation.
• Easy Continuous Integration through 3rd party Open Source libraries for Maven, Graddle and Jenkins

JMeter is not a browser

JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it’s possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).

Running JMeter

To run JMeter, run the jmeter.bat (for Windows) or jmeter (for Unix) file. These files are found in the bin directory. After a short time, the JMeter GUI should appear. GUI mode should only be used for creating the test script, NON GUI mode must be used for load testing.

There are some additional scripts in the bin directory that you may find useful. Windows script files (the .CMD files require Win2K or later):

• bat – run JMeter (in GUI mode by default)
• cmd – run JMeter without the windows shell console (in GUI mode by default)
• jmeter-n.cmd – drop a JMX file on this to run a non-GUI test
• jmeter-n-r.cmd – drop a JMX file on this to run a non-GUI test remotely
• jmeter-t.cmd – drop a JMX file on this to load it in GUI mode
• jmeter-server.bat – start JMeter in server mode
• mirror-server.cmd – runs the JMeter Mirror Server in non-GUI mode
• cmd – Run the Shutdown client to stop a non-GUI instance gracefully
• cmd – Run the Shutdown client to stop a non-GUI instance abruptly

The special name LAST can be used with jmeter-n.cmd, jmeter-t.cmd and jmeter-n-r.cmd and means the last test plan that was run interactively. There are a few environment variables, that can be used to customize the JVM settings for JMeter. An easy way to set those is by creating a file named setenv.bat in the bin directory. Such a file could look like:

rem This is the content of bin\setenv.bat,

rem it will be called by bin\jmeter.bat

set JVM_ARGS=”-Xms1024m -Xmx1024m -Dpropname=value”

The JVM_ARGS can be used to override JVM settings in the jmeter.bat script and will get set when starting JMeter,

jmeter -t test.jmx …

The following environment variables can be defined:

• DDRAW – JVM options to influence usage of direct draw, e.g. -Dsun.java2d.ddscale=true. Default is empty.
• GC_ALGO – JVM garbage collector options. Defaults to -XX:+UseG1GC -XX:MaxGCPauseMillis=250 -XX:G1ReservePercent=20
• HEAP – JVM memory settings used when starting JMeter. Defaults to -Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
• JMETER_BIN – JMeter bin directory (must end in \). Value will have been guessed, when setenv.bat is called.
• JMETER_COMPLETE_ARGS – If set indicates, that JVM_ARGS and JMETER_OPTS are to be used, only. All other options like HEAP and GC_ALGO will be ignored. Default is empty.
• JMETER_HOME – installation directory. Will be guessed from location of jmeter.bat
• JMETER_LANGUAGE – Java runtime options to specify used languauge. Defaults to: -Duser.language=”en” -Duser.region=”EN”
• JM_LAUNCH – Name of the java executable, like java.exe (default) or javaw.exe
• JVM_ARGS – Java options to be used when starting JMeter. These will be added last to the java command. Default is empty

Un*x script files; should work on most Linux/Unix systems:

• jmeter – run JMeter (in GUI mode by default). Defines some JVM settings which may not work for all JVMs.
• jmeter-server – start JMeter in server mode (calls jmeter script with appropriate parameters)
• sh – very basic JMeter script (You may need to adapt JVM options like memory settings).
• mirror-server.sh – runs the JMeter Mirror Server in non-GUI mode
• sh – Run the Shutdown client to stop a non-GUI instance gracefully
• sh – Run the Shutdown client to stop a non-GUI instance abruptly

It may be necessary to set a few environment variables to configure the JVM used by JMeter. Those variables can be either set directly in the shell starting the jmeter script. For example setting the variable JVM_ARGS will override most pre-defined settings, for example

JVM_ARGS=”-Xms1024m -Xmx1024m” jmeter -t test.jmx [etc.]

will override the HEAP settings in the script.

To set those variables permanently, you can place them in a file called setenv.sh in the bin directory. This file will be sourced when running JMeter by calling the jmeter script. An example for bin/setenv.sh could look like:

# This is the file bin/setenv.sh,

# it will be sourced in by bin/jmeter

# Use a bigger heap, but a smaller metaspace, than the default

export HEAP=”-Xms1G -Xmx1G -XMaxMetaspaceSize=192m”

# Try to guess the locale from the OS. The space as value is on purpose!

export JMETER_LANGUAGE=” ”

The following environment variables can be defined:

• GC_ALGO – Java runtime options to specify JVM garbage collection algorithm. Defaults to -XX:+UseG1GC -XX:MaxGCPauseMillis=250 -XX:G1ReservePercent=20
• HEAP – Java runtime options for memory management used when JMeter is started. Defaults to -Xms1g -Xmx1g -X:MaxMetaspaceSize=256m
• JAVA_HOME – Must point at your Java Development Kit installation. Required to run the with the “debug” argument. On some OSes it JMeter will try its best to guess the location of the JVM.
• JMETER_COMPLETE_ARGS – If set indicates, that JVM_ARGS and JMETER_OPTS are to be used, only. All other options like HEAP and GC_ALGO will be ignored. Default is empty.
• JMETER_HOME – May point to your JMeter install dir. If empty it will be set relative to the jmeter script.
• JMETER_LANGUAGE – Java runtime options to specify used language. Defaults to -Duser.language=en -Duser.region=EN
• JMETER_OPTS – Java runtime options used when JMeter is started. Special options for operating systems might be added by JMeter.
• JRE_HOME – Must point at your Java Runtime installation. Defaults to JAVA_HOME if empty. If JRE_HOME and JAVA_HOME are both empty, JMeter will try to guess JAVA_HOME. If JRE_HOME and JAVA_HOME are both set, JAVA_HOME is used.
• JVM_ARGS – Java options to be used when starting JMeter. These will be added before JMETER_OPTS and after the other JVM options. Default is empty

JMeter’s Classpath

JMeter automatically finds classes from jars in the following directories:

• JMETER_HOME/lib – used for utility jars
• JMETER_HOME/lib/ext – used for JMeter components and plugins

If you have developed new JMeter components, then you should jar them and copy the jar into JMeter’s lib/ext directory. JMeter will automatically find JMeter components in any jars found here. Do not use lib/ext for utility jars or dependency jars used by the plugins; it is only intended for JMeter components and plugins.

If you don’t want to put JMeter plugin jars in the lib/ext directory, then define the property search_paths in jmeter.properties.  Utility and dependency jars (libraries etc) can be placed in the lib directory. If you don’t want to put such jars in the lib directory, then define the property user.classpath or plugin_dependency_paths in jmeter.properties.

Other jars (such as JDBC, JMS implementations and any other support libraries needed by the JMeter code) should be placed in the lib directory – not the lib/ext directory, or added to user.classpath. JMeter will only find .jar files, not .zip.

You can also install utility Jar files in $JAVA_HOME/jre/lib/ext, or you can set the property user.classpath in jmeter.properties Setting the CLASSPATH environment variable will have no effect. This is because JMeter is started with “java -jar”, and the java command silently ignores the CLASSPATH variable, and the -classpath/-cp options when -jar is used. This occurs with all Java programs, not just JMeter.

Create Test Plan from Template

You have the ability to create a new Test Plan from existing template. To do so you use the menu File → Templates… or Templates icon:

A popup appears, you can then choose a template among the list:

A documentation for each template explains what to do once test plan is created from template.

Full list of command-line options

Invoking JMeter as “jmeter -?” will print a list of all the command-line options. These are shown below.

–?

print command line options and exit

-h, –help

print usage information and exit

-v, –version

print the version information and exit

-p, –propfile <argument>

the jmeter property file to use

-q, –addprop <argument>

additional JMeter property file(s)

-t, –testfile <argument>

the jmeter test(.jmx) file to run

-l, –logfile <argument>

the file to log samples to

-i, –jmeterlogconf <argument>

jmeter logging configuration file (log4j2.xml)

-j, –jmeterlogfile <argument>

jmeter run log file (jmeter.log)

-n, –nongui

run JMeter in nongui mode

-s, –server

run the JMeter server

-H, –proxyHost <argument>

Set a proxy server for JMeter to use

-P, –proxyPort <argument>

Set proxy server port for JMeter to use

-N, –nonProxyHosts <argument>

Set nonproxy host list (e.g. *.apache.org|localhost)

-u, –username <argument>

Set username for proxy server that JMeter is to use

-a, –password <argument>

Set password for proxy server that JMeter is to use

-J, –jmeterproperty <argument>=<value>

Define additional JMeter properties

-G, –globalproperty <argument>=<value>

Define Global properties (sent to servers)

e.g. -Gport=123

or -Gglobal.properties

-D, –systemproperty <argument>=<value>

Define additional system properties

-S, –systemPropertyFile <argument>

additional system property file(s)

-f, –forceDeleteResultFile

force delete existing results files before start the test

-L, –loglevel <argument>=<value>

[category=]level e.g. jorphan=INFO, jmeter.util=DEBUG or com.example.foo=WARN

-r, –runremote

Start remote servers (as defined in remote_hosts)

-R, –remotestart <argument>

Start these remote servers (overrides remote_hosts)

-d, –homedir <argument>

the jmeter home directory to use

-X, –remoteexit

Exit the remote servers at end of test (non-GUI)

-g, –reportonly <argument>

generate report dashboard only, from a test results file

-e, –reportatendofloadtests

generate report dashboard after load test

-o, –reportoutputfolder <argument>

output folder for report dashboard

The JMeter log file name is formatted as a SimpleDateFormat (applied to the current date) if it contains paired single-quotes, .e.g. ‘jmeter_’yyyyMMddHHmmss’.log’

If the special name LAST is used for the -t, -j or -l flags, then JMeter takes that to mean the last test plan that was run in interactive mode.