This document describes the results of performance tests for WebSpellChecker Web API. The performance was tested and accessed depending on the following setup:

  • Certain number of users accessing the server simultaneously;
  • Cache enabled for spell checking purposes; 
  • Certain hardware and software used; 
  • Number of words to be checked;
  • Number of spelling and grammar problems in the text.

Testing goal and idea

Our main goal was to observe the average response time of text processing in case when 10/20/50/100/200 users send simultaneous requests to the server between two different versions of WebSpellChecker, namely 5.5.3 and 5.5.4.

Environment and testing tool

We used Apache Jmeter 5.1.1 as a performance measuring tool. The machine used was Amazon AWS m5.large instance with 2 vCPU and 8 GB RAM.

Testing process

We have run our tests 5 times continuously increasing the number of users accessing them. In every test the number of simultaneous users increased by 10, thus, making 5 test cases increasing workload from 10 to 50 users or simultaneous threads. The cache setting was enabled for the first set of tests and disabled later. The tests took place in the following order:

  1. 10K words (6K characters) with 10 grammar problems, 50 misspellings;
  2. 10K words (6K characters) with 15 grammar problems, 250 misspellings;
  3. 10K words (6K characters) with 30 grammar problems, 400 misspellings;
  4. 10K words with 50 misspellings only;
  5. 10K words with 50 grammar problems only.

We used the concept of tokens which is a complete sentence to be spell and grammar checked. In the test setup, we had 10 tokens. 

The measured was the response time for versions 5.5.3 and 5.5.4 respectively.

Observations and findings

Our observations are presented in tables and charts below for WebSpellChecker Server versions 5.5.3 and 5.5.4.

  • The average time for processing of 1K words containing words in mixed case and misspelled words only grows 15-25% in version 5.5.4 depending on the number of users compared with version 5.5.3, 
  • If users need to proofread 1K words containing only grammar problems, and there are no misspelled words in these sentences, it will take more time, namely 27,777 seconds in version 5.5.4x compared with 46,551 seconds in version 5.5.5x. For details, see section 1K words with 50 grammar problems only depending on the number of users below.
Disabled cache setting, which can be specified as the value of the CacheSize parameter in AppServerX.xml file, is a standard scenario of WebSpellChecker setup.

1st test case for 10 simultaneous users


WSC 5.5.3, response in seconds

WSC 5.5.4 - 10 tokens, response in seconds
Cache disabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

3,680

2,706

1K words (6K chars) with 15 grammar problems 250 misspellings

10,378

10,412

1K words (6K chars) with 30 grammar problems 400 misspellings

14,478

13,869

1K words with 50 misspellings only

3,976

3,666

1K words with 50 grammar problems only

1,668

2,367

Cache enabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

3,124

1,918

1K words (6K chars) with 15 grammar problems 250 misspellings

3,115

2,034

1K words (6K chars) with 30 grammar problems 400 misspellings

3,802

1,855

1K words with 50 misspellings only

3,687

2,303

Chart below represents test results comparison for WSC v.5.3 and WSC v.5.4 and 10 simultaneous users with disabled cache.

2nd test case for 20 simultaneous users


WSC 5.5.3, response in seconds

WSC 5.5.4, 10 tokens, response in seconds
Cache disabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

10,229

8,155

1K words (6K chars) with 15 grammar problems 250 misspellings

25,387

24,795

1K words (6K chars) with 30 grammar problems 400 misspellings

35,331

33,962

1K words with 50 misspellings only

12,521

8,456

1K words with 50 grammar problems only

3,160

4,261

Cache enabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

5,275

3,599

1K words (6K chars) with 15 grammar problems 250 misspellings

5,252

3,613

1K words (6K chars) with 30 grammar problems 400 misspellings

5,403

3,125

1K words with 50 misspellings only

7,959

3,855

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 20 simultaneous users with disabled cache.

3rd test case for 50 simultaneous users


WSC 5.5.3, response in seconds

WSC 5.5.4, 10 tokens, response in seconds

Cache disabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

34,262

29,289

1K words (6K chars) with 15 grammar problems 250 misspellings

74,255

72,647

1K words (6K chars) with 30 grammar problems 400 misspellings

95,067

95,850

1K words with 50 misspellings only

36,474

29,277

1K words with 50 grammar problems only

12,094

19,454

Cache enabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

21,304

17,645

1K words (6K chars) with 15 grammar problems 250 misspellings

22,008

17,885

1K words (6K chars) with 30 grammar problems 400 misspellings

23,647

17,539

1K words with 50 misspellings only

24,189

17,910

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 50 simultaneous users with disabled cache.

4th test case for 100 simultaneous users


WSC 5.5.3, response in seconds

WSC 5.5.4, 10 tokens, response in seconds

Cache disabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

75,619

66,855

1K words (6K chars) with 15 grammar problems 250 misspellings

157,668

155,222

1K words (6K chars) with 30 grammar problems 400 misspellings

198,941

199,314

1K words with 50 misspellings only

80,780

67,456

1K words with 50 grammar problems only

27,777

46,551

Cache enabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

50,682

43,647

1K words (6K chars) with 15 grammar problems 250 misspellings

52,005

43,139

1K words (6K chars) with 30 grammar problems 400 misspellings

53,959

43,366

1K words with 50 misspellings only

55,802

43,614

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 100 simultaneous users with disabled cache.

5th test case for 200 simultaneous users


WSC 5.5.3, response in secondsWSC 5.5.4, 10 tokens, response in seconds

Cache disabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

160,952

144,106

1K words (6K chars) with 15 grammar problems 250 misspellings

323,478

314,992

1K words (6K chars) with 30 grammar problems 400 misspellings

409,702

407,870

1K words with 50 misspellings only

170,428

144,126

1K words with 50 grammar problems only

61,754

102,238

Cache enabled for spelling errors

1K words (6K chars) with 10 grammar problems 50 misspellings

110,704

97,814

1K words (6K chars) with 15 grammar problems 250 misspellings

113,615

97,839

1K words (6K chars) with 30 grammar problems 400 misspellings

113,767

102,991

1K words with 50 misspellings only

120,933

95,217

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 and 200 simultaneous users with disabled cache.

 1K words (6K chars) with 10 grammar problems 50 misspellings depending on the number of users

Number of usersWSC 5.5.3, response in secondsWSC 5.5.4, response in seconds
10

3,680

2,706

20

10,229

8,155

50

34,262

29,289

100

75,619

66,855

200

160,952

144,106

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 10 grammar problems and 50 misspellings scenario.

1K words (6K chars) with 15 grammar problems 250 misspellings depending on the number of users

Number of usersWSC 5.5.3, response in secondsWSC 5.5.4, response in seconds
10

10,378

10,412

20

25,387

24,795

50

74,255

72,647

100

157,668

155,222

200

323,478

314,992

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 15 grammar problems and 250 misspellings scenario.

1K words (6K chars) with 30 grammar problems 400 misspellings depending on the number of users

Number of usersWSC 5.5.3, response in secondsWSC 5.5.4, response in seconds
10

14,478

13,869

20

35,331

33,962

50

95,067

95,850

100

198,941

199,314

200

409,702

407,870

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 30 grammar problems and 400 misspellings scenario.

1K words with 50 misspellings only depending on the number of users

Number of usersWSC 5.5.3, response in secondsWSC 5.5.4, response in seconds
10

3,976

3,666

20

12,521

8,456

50

36,474

29,277

100

80,780

67,456

200

170,428

144,126

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 misspellings scenario.

1K words with 50 grammar problems only depending on the number of users

Number of usersWSC 5.5.3, response in secondsWSC 5.5.4, response in seconds
10

1,668

2,367

20

3,160

4,261

50

12,094

19,454

100

27,777

46,551

200

61,754

102,238

Chart below represents test result comparison for WSC v.5.3 and WSC v.5.4 for different number of users in 50 grammar problems scenario.

Recommendations 

Here are the outcomes and aftermath as well as our advice on hardware and software requirements and notes on performance issues which users may encounter:

  • General performance of our spell check engine has increased, but grammar engine performance is not as high as expected. For details, see the charts showing test results depending on the number of users. For example, 1K words (6K chars) with 15 grammar problems 250 misspellings and other graphs in this section.
  • 1 m5 instance can process 150-200 simultaneous users, or simultaneous threads, without any issues, but when the number of users increases to 200+, it entails 100% CPU load and a significant increase of response time. Our recommendation for he case when more users are added and CPU load constantly reaches 100% on the machine:
    • upgrade instance type and add more CPUs to it;
    • add one more machine to distribute the traffic (requests) between two or more machines, for example, using Load Balancer.

When cache is enabled, tests run much faster, and the results are almost identical for different cases due to the processed texts are the same. This case needs refinement of the text uniqueness or some text randomising for each request being sent. Our recommendation for this case is the following: specify the desired value of the CacheSize parameter in AppServerX.xml file to increase the speed of requests processing.


  • No labels