This document describes the results of performance tests for WebSpellChecker Web API. The performance was tested and accessed depending on the following setup:
Our main goal was to observe the average response time of text processing in case when 10/20/50/100/200 users send simultaneous requests to the server between two different versions of WebSpellChecker, namely 5.5.3 and 5.5.4.
We used Apache Jmeter 5.1.1 as a performance measuring tool. The machine used was Amazon AWS m5.large instance with 2 vCPU and 8 GB RAM.
We have run our tests 5 times continuously increasing the number of users accessing them. In every test the number of simultaneous users increased by 10, thus, making 5 test cases increasing workload from 10 to 50 users or simultaneous threads. The cache setting was enabled for the first set of tests and disabled later. The tests took place in the following order:
We used the concept of tokens which is a complete sentence to be spell and grammar checked. In the test setup, we had 10 tokens.
The measured was the response time for versions 5.5.3 and 5.5.4 respectively.
Our observations are presented in tables and charts below for WebSpellChecker Server versions 5.5.3 and 5.5.4.
Disabled cache setting, which can be specified as the value of the CacheSize parameter in AppServerX.xml file, is a standard scenario of WebSpellChecker setup. |
WSC 5.5.3, response in seconds | WSC 5.5.4 - 10 tokens, response in seconds | |
---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 3,680 | 2,706 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 10,378 | 10,412 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 14,478 | 13,869 |
1K words with 50 misspellings only | 3,976 | 3,666 |
1K words with 50 grammar problems only | 1,668 | 2,367 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 3,124 | 1,918 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 3,115 | 2,034 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 3,802 | 1,855 |
1K words with 50 misspellings only | 3,687 | 2,303 |
Chart below represents test results comparison for WSC v.5.3 and WSC v.5.4 and 10 simultaneous users with disabled cache.
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 10,229 | 8,155 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 25,387 | 24,795 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 35,331 | 33,962 |
1K words with 50 misspellings only | 12,521 | 8,456 |
1K words with 50 grammar problems only | 3,160 | 4,261 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 5,275 | 3,599 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 5,252 | 3,613 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 5,403 | 3,125 |
1K words with 50 misspellings only | 7,959 | 3,855 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 20 simultaneous users with disabled cache.
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 34,262 | 29,289 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 74,255 | 72,647 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 95,067 | 95,850 |
1K words with 50 misspellings only | 36,474 | 29,277 |
1K words with 50 grammar problems only | 12,094 | 19,454 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 21,304 | 17,645 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 22,008 | 17,885 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 23,647 | 17,539 |
1K words with 50 misspellings only | 24,189 | 17,910 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 50 simultaneous users with disabled cache.
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 75,619 | 66,855 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 157,668 | 155,222 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 198,941 | 199,314 |
1K words with 50 misspellings only | 80,780 | 67,456 |
1K words with 50 grammar problems only | 27,777 | 46,551 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 50,682 | 43,647 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 52,005 | 43,139 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 53,959 | 43,366 |
1K words with 50 misspellings only | 55,802 | 43,614 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 100 simultaneous users with disabled cache.
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 160,952 | 144,106 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 323,478 | 314,992 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 409,702 | 407,870 |
1K words with 50 misspellings only | 170,428 | 144,126 |
1K words with 50 grammar problems only | 61,754 | 102,238 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 110,704 | 97,814 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 113,615 | 97,839 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 113,767 | 102,991 |
1K words with 50 misspellings only | 120,933 | 95,217 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 200 simultaneous users with disabled cache.
Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
---|---|---|
10 | 3,680 | 2,706 |
20 | 10,229 | 8,155 |
50 | 34,262 | 29,289 |
100 | 75,619 | 66,855 |
200 | 160,952 | 144,106 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 10 grammar problems and 50 misspellings scenario.
Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
---|---|---|
10 | 10,378 | 10,412 |
20 | 25,387 | 24,795 |
50 | 74,255 | 72,647 |
100 | 157,668 | 155,222 |
200 | 323,478 | 314,992 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 15 grammar problems and 250 misspellings scenario.
Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
---|---|---|
10 | 14,478 | 13,869 |
20 | 35,331 | 33,962 |
50 | 95,067 | 95,850 |
100 | 198,941 | 199,314 |
200 | 409,702 | 407,870 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 30 grammar problems and 400 misspellings scenario.
Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
---|---|---|
10 | 3,976 | 3,666 |
20 | 12,521 | 8,456 |
50 | 36,474 | 29,277 |
100 | 80,780 | 67,456 |
200 | 170,428 | 144,126 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 misspellings scenario.
Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
---|---|---|
10 | 1,668 | 2,367 |
20 | 3,160 | 4,261 |
50 | 12,094 | 19,454 |
100 | 27,777 | 46,551 |
200 | 61,754 | 102,238 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 grammar problems scenario.
Here are the outcomes and aftermath as well as our advice on hardware and software requirements and notes on performance issues which users may encounter:
When cache is enabled, tests run much faster, and the results are almost identical for different cases due to the processed texts are the same. This case needs refinement of the text uniqueness or some text randomising for each request being sent. Our recommendation for this case is the following: specify the desired value of the CacheSize parameter in AppServerX.xml file to increase the speed of requests processing.