Page History
...
- Certain number of users accessing the server simultaneously;
- Cache enabled for spell checking purposes;
- Certain hardware and software used;
- Number of words to be checked;
- Number of spelling and grammar problems in the text.
Testing
...
goal and
...
idea
Our main goal was to observe the average response time of text processing in case when 10/20/50/100/200 users send simultaneous requests to the server between two different versions of WebSpellChecker, namely 5.5.3 and 5.5.4.
Environment and
...
testing tool
We used Apache Jmeter 5.1.1 as a performance measuring tool. The machine used was Amazon AWS m5.large instance with 2 vCPU and 8 GB RAM.
Testing
...
process
We have run our tests 5 times continuously increasing the number of users accessing itthem. In every test the number of simultaneous users increased by 10, thus, making 5 test cases increasing workload from 10 to 50 users or simultaneous threads. The cache setting was enabled for the first set of tests and disabled later. The tests took place in the following order:
- 10K words (6K characters) with 10 grammar problems, 50 misspellings;
- 10K words (6K characters) with 15 grammar problems, 250 misspellings;
- 10K words (6K characters) with 30 grammar problems, 400 misspellings;
- 10K words with 50 misspellings only;
- 10K words with 50 grammar problems only.
...
The measured was the response time for versions 5.5.3 and 5.5.4 respectively.
Observations and
...
findings
Our observations are presented in tables and charts below for WebSpellChecker Server versions 5.5.3 and 5.5.4.
...
| Note |
|---|
| Disabled cache setting, which can be specified as the value of the CacheSize parameter in AppServerX.xml file, is a standard scenario of WebSpellChecker setup. |
1st
...
test case for 10
...
simultaneous users
WSC 5.5.3, response in seconds | WSC 5.5.4 - 10 tokens, response in seconds | |
|---|---|---|
| Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 3,680 | 2,706 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 10,378 | 10,412 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 14,478 | 13,869 |
1K words with 50 misspellings only | 3,976 | 3,666 |
1K words with 50 grammar problems only | 1,668 | 2,367 |
| Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 3,124 | 1,918 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 3,115 | 2,034 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 3,802 | 1,855 |
1K words with 50 misspellings only | 3,687 | 2,303 |
Chart below represents test results comparison for WSC v.5.3 and WSC v.5.4 and 10 simultaneous users with disabled cache.
2nd
...
test case for 20 simultaneous users
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
|---|---|---|
| Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 10,229 | 8,155 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 25,387 | 24,795 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 35,331 | 33,962 |
1K words with 50 misspellings only | 12,521 | 8,456 |
1K words with 50 grammar problems only | 3,160 | 4,261 |
| Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 5,275 | 3,599 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 5,252 | 3,613 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 5,403 | 3,125 |
1K words with 50 misspellings only | 7,959 | 3,855 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 20 simultaneous users with disabled cache.
3rd
...
test case for 50 simultaneous users
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
|---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 34,262 | 29,289 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 74,255 | 72,647 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 95,067 | 95,850 |
1K words with 50 misspellings only | 36,474 | 29,277 |
1K words with 50 grammar problems only | 12,094 | 19,454 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 21,304 | 17,645 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 22,008 | 17,885 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 23,647 | 17,539 |
1K words with 50 misspellings only | 24,189 | 17,910 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 50 simultaneous users with disabled cache.
4th
...
test case for 100
...
simultaneous users
WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
|---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 75,619 | 66,855 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 157,668 | 155,222 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 198,941 | 199,314 |
1K words with 50 misspellings only | 80,780 | 67,456 |
1K words with 50 grammar problems only | 27,777 | 46,551 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 50,682 | 43,647 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 52,005 | 43,139 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 53,959 | 43,366 |
1K words with 50 misspellings only | 55,802 | 43,614 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 100 simultaneous users with disabled cache.
5th
...
test case for 200 simultaneous users
| WSC 5.5.3, response in seconds | WSC 5.5.4, 10 tokens, response in seconds | |
|---|---|---|
Cache disabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 160,952 | 144,106 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 323,478 | 314,992 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 409,702 | 407,870 |
1K words with 50 misspellings only | 170,428 | 144,126 |
1K words with 50 grammar problems only | 61,754 | 102,238 |
Cache enabled for spelling errors | ||
1K words (6K chars) with 10 grammar problems 50 misspellings | 110,704 | 97,814 |
1K words (6K chars) with 15 grammar problems 250 misspellings | 113,615 | 97,839 |
1K words (6K chars) with 30 grammar problems 400 misspellings | 113,767 | 102,991 |
1K words with 50 misspellings only | 120,933 | 95,217 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 200 simultaneous users with disabled cache.
1K words (6K chars) with 10 grammar problems 50 misspellings depending on the number of users
| Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
|---|---|---|
| 10 | 3,680 | 2,706 |
| 20 | 10,229 | 8,155 |
| 50 | 34,262 | 29,289 |
| 100 | 75,619 | 66,855 |
| 200 | 160,952 | 144,106 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 10 grammar problems and 50 misspellings scenario.
1K words (6K chars) with 15 grammar problems 250 misspellings depending on the number of users
| Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
|---|---|---|
| 10 | 10,378 | 10,412 |
| 20 | 25,387 | 24,795 |
| 50 | 74,255 | 72,647 |
| 100 | 157,668 | 155,222 |
| 200 | 323,478 | 314,992 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 15 grammar problems and 250 misspellings scenario.
1K words (6K chars) with 30 grammar problems 400 misspellings depending on the number of users
| Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
|---|---|---|
| 10 | 14,478 | 13,869 |
| 20 | 35,331 | 33,962 |
| 50 | 95,067 | 95,850 |
| 100 | 198,941 | 199,314 |
| 200 | 409,702 | 407,870 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 30 grammar problems and 400 misspellings scenario.
1K words with 50 misspellings only depending on the number of users
| Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
|---|---|---|
| 10 | 3,976 | 3,666 |
| 20 | 12,521 | 8,456 |
| 50 | 36,474 | 29,277 |
| 100 | 80,780 | 67,456 |
| 200 | 170,428 | 144,126 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 misspellings scenario.
1K words with 50 grammar problems only depending on the number of users
| Number of users | WSC 5.5.3, response in seconds | WSC 5.5.4, response in seconds |
|---|---|---|
| 10 | 1,668 | 2,367 |
| 20 | 3,160 | 4,261 |
| 50 | 12,094 | 19,454 |
| 100 | 27,777 | 46,551 |
| 200 | 61,754 | 102,238 |
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 grammar problems scenario.
Recommendations
Here are the outcomes and aftermath as well as our advice on hardware and software requirements and notes on performance issues which users may encounter:
...
