Page History
This document describes the results of performance tests for WebSpellChecker Web API. The performance was and load were tested and accessed depending on the following setup:
- Certain number of users accessing the server simultaneously (10/20/50/100 users);
- Cache enabled for spell checking purposes;
- Certain hardware and software used (EC2 m5.large instance with 2 vCPU and 8 GB RAM);
- Number of words to be checked (10K words or 6K characters);
- Number of spelling and grammar problems in the text (50 grammar problems | 200 misspellings);
- Type of the language used for check (17 default languages).
Testing Goal and Idea
Our main goal was to observe the response time of text processing and CPU utilization on the server in case when 10/20/50/100 users send simultaneous requests on various languages to the server with WebSpellChecker v5.5.9.
Environment and Testing Tool
- We used Apache Jmeter 5.1.1 as a performance measuring tool.
- The machine used was
...
- AWS EC2 m5.large instance with 2 vCPU and 8 GB RAM.
- The version of the WebSpellChecker Server package is 5.5.9 (released on April 21, 2020).
Testing Process
We have run our tests 4 times continuously increasing the number of users accessing it for each of the languages in the default language group . In every test the number of simultaneous users increased by 10, thus, making 5 test cases increasing workload from 10 to 50 users or simultaneous threads. (17 languages). The cache setting was enabled for the first all set of tests and disabled later. The tests took place in the following order:
- 10K words (6K characters) with 50 grammar problems 200 misspellings;
- 10K words with 200 misspellings 50 grammar problems only;
- 10K words with 50 grammar problems 200 misspellings only.
We used the concept of tokens which is a complete sentence to be spell and grammar checked. In the test setup, we had 10 tokens.
The measured was the response time for version 5.5.9 respectively.and CPU utilization.
Observations and Findings
Our observations are presented in tables and charts below for WebSpellChecker Server versions v5.5.5.3 and 5.5.49.
- The response time and CPU untilization increase as more simultaneous users are added.
- These two metrics depend a lot on the number of words in the dictionary for spelling check and number of rules for grammar check.
- The average time for processing of 1K words containing words in mixed case and misspelled words only grows 15-25% in version 5.5.4 depending on the number of users compared with version 5.5.3,
- If users need to proofread 1K words containing only grammar problems, and there are no misspelled words in these sentences, it will take more time, namely 27,777 seconds in version 5.5.4x compared with 46,551 seconds in version 5.5.5x. For details, see section 1K words with 50 grammar problems only depending on the number of users below.
| Note |
|---|
| Disabled cache setting, which can be specified as the value of the CacheSize parameter in AppServerX.xml file, is a standard scenario of WebSpellChecker setup. |
1st Test Case for 10 Simultaneous Users
...
WSC 5.5.3, response in seconds
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
3,680
...
2,706
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
10,378
...
10,412
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
14,478
...
13,869
...
1K words with 50 misspellings only
...
3,976
...
3,666
...
1K words with 50 grammar problems only
...
1,668
...
2,367
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
3,124
...
1,918
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
3,115
...
2,034
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
...
1,855
...
1K words with 50 misspellings only
...
3,687
...
2,303
Chart below represents test results
...
2nd Test Case for 20 Simultaneous Users
...
WSC 5.5.3, response in seconds
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
10,229
...
8,155
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
25,387
...
24,795
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
35,331
...
33,962
...
1K words with 50 misspellings only
...
12,521
...
8,456
...
1K words with 50 grammar problems only
...
3,160
...
4,261
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
5,275
...
3,599
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
5,252
...
3,613
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
5,403
...
3,125
...
1K words with 50 misspellings only
...
7,959
...
3,855
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 20 simultaneous users with disabled cache.
3rd Test Case for 50 Simultaneous Users
...
WSC 5.5.3, response in seconds
...
Cache disabled for spelling errors
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
34,262
...
29,289
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
74,255
...
72,647
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
95,067
...
95,850
...
1K words with 50 misspellings only
...
36,474
...
29,277
...
1K words with 50 grammar problems only
...
12,094
...
19,454
...
Cache enabled for spelling errors
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
21,304
...
17,645
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
22,008
...
17,885
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
23,647
...
17,539
...
1K words with 50 misspellings only
...
24,189
...
17,910
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 50 simultaneous users with disabled cache.
4th Test Case for 100 Simultaneous Users
...
WSC 5.5.3, response in seconds
...
Cache disabled for spelling errors
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
75,619
...
66,855
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
157,668
...
155,222
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
198,941
...
199,314
...
1K words with 50 misspellings only
...
80,780
...
67,456
...
1K words with 50 grammar problems only
...
27,777
...
46,551
...
Cache enabled for spelling errors
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
50,682
...
43,647
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
52,005
...
43,139
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
53,959
...
43,366
...
1K words with 50 misspellings only
...
55,802
...
43,614
...
comparison for WSC v.5.3 and WSC v.5.4 and
...
5th Test Case for 200 Simultaneous Users
...
Cache disabled for spelling errors
...
1K words (6K chars) with 10 grammar problems 50 misspellings
...
160,952
...
144,106
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
323,478
...
314,992
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
409,702
...
407,870
...
1K words with 50 misspellings only
...
170,428
...
144,126
...
1K words with 50 grammar problems only
...
61,754
...
102,238
...
Cache enabled for spelling errors
...
10
...
110,704
...
97,814
...
1K words (6K chars) with 15 grammar problems 250 misspellings
...
113,615
...
97,839
...
1K words (6K chars) with 30 grammar problems 400 misspellings
...
113,767
...
102,991
...
1K words with 50 misspellings only
...
120,933
...
95,217
...
simultaneous users with disabled cache
...
1K words (6K chars) with 10 grammar problems 50 misspellings depending on the number of users
...
3,680
...
2,706
...
10,229
...
8,155
...
34,262
...
29,289
...
75,619
...
66,855
...
160,952
...
144,106
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 10 grammar problems and 50 misspellings scenario.
1K words (6K chars) with 15 grammar problems 250 misspellings depending on the number of users
...
10,378
...
10,412
...
25,387
...
24,795
...
74,255
...
72,647
...
157,668
...
155,222
...
323,478
...
314,992
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 15 grammar problems and 250 misspellings scenario.
1K words (6K chars) with 30 grammar problems 400 misspellings depending on the number of users
...
14,478
...
13,869
...
35,331
...
33,962
...
95,067
...
95,850
...
198,941
...
199,314
...
409,702
...
407,870
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 30 grammar problems and 400 misspellings scenario.
1K words with 50 misspellings only depending on the number of users
...
3,976
...
3,666
...
12,521
...
8,456
...
36,474
...
29,277
...
80,780
...
67,456
...
170,428
...
144,126
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 misspellings scenario.
1K words with 50 grammar problems only depending on the number of users
...
.
...
...
1,668
...
2,367
...
3,160
...
4,261
...
12,094
...
19,454
...
27,777
...
46,551
...
61,754
...
102,238
Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 grammar problems scenario.
Recommendations
Here are the outcomes and aftermath as well as our advice on hardware and software requirements and notes on performance issues which users may encounter:
- General performance of our spell check engine has increased, but grammar engine performance is not as high as expected. For details, see the charts showing test results depending on the number of users. For example, 1K words (6K chars) with 15 50 grammar problems 250 200 misspellings and other graphs in this section.
- 1 One m5 instance can process 150-200 simultaneous users, or simultaneous threads, without any issues, but when the number of users increases to 200+, it entails 100% CPU load and a significant increase of response time. Our recommendation for he the case when more users are added and CPU load constantly reaches 100% on the machine:
- upgrade instance type and add more CPUs to it;
- add one more machine to distribute the traffic (requests) between two or more machines, for example, using Load Balancerload balance and auto-scalling.
When cache is enabled, tests run much faster, and the results are almost identical for different cases due to the processed texts are the same. This case needs refinement of the text uniqueness or some text randomising for each request being sent. Our recommendation for this case is the following: specify the desired value of the CacheSize parameter in AppServerX.xml file to increase the speed of requests processing.
...
