REDIS Stress Test Report – Phase I

Scope

This report describes the test strategy and results of testing the completeness of REDIS DB Server implementation for Windows.

Test Coverage

• Redis servers:

• version 2.4

• 32bit

• 64bit

• version 2.6

• 32bit

• 64bit

• All value types including:

• Strings

• Lists

• Sets

• Hashes

• Sorted sets

• Client-server configurations:

• 1 master - 0 slaves - 4 clients (4 clients per server)

• 1 master - 2 slaves - 4 clients (2 clients per server)

• 1 master - 4 slaves - 4 clients (1 client per server)

Testing Procedures

Generally procedure can be described as follows:

• While version < max
  • Take REDIS Server of correspondent version
  • While configuration setup < max
  • Deploy one of the client-server configuration
  • Fill database with initial values (7000 of each value type)
  • Verify that data added to the database without problems
  • Store number of values added and value of memory used field
  • Run the automated test for specified time
  • Compare number of values added and value of memory used field with initial
  • Check logs for errors
  • Check process memory usage
  • Loop
• Loop

Automation Scenario

##Client 1

• Add 5000 sets
• Add 5000 strings
• SYNC
• Remove 5000 strings
• SYNC
• Add 5000 sorted sets
• SYNC
• Add 5000 strings
• Remove 5000 sets
• SYNC
• Remove 5000 lists
• SYNC
• Remove 5000 hashes
• SYNC
• Add 5000 sorted sets

Client 2

• Add 5000 lists
• Remove 5000 lists
• SYNC
• Add 5000 hashes
• SYNC
• Add 5000 lists
• SYNC
• Remove 5000 strings
• SYNC
• Remove 5000 sets
• SYNC
• Remove 5000 sorted sets
• SYNC
• Add 5000 hashes

Client 3

• Add 5000 hashes
• Remove 5000 hashes
• SYNC
• Remove 5000 sets
• SYNC
• Add 5000 sets
• SYNC
• Remove 5000 hashes
• SYNC
• Add 5000 sorted sets
• SYNC
• Add 5000 strings
• SYNC
• Remove 5000 lists
• Remove 5000 hashes

Client 4

• Add 5000 sorted sets
• Remove 5000 sorted sets
• SYNC
• Remove 5000 sorted sets
• SYNC
• Add 5000 sets
• SYNC
• Add 5000 hashes
• SYNC
• Add 5000 lists
• SYNC
• Remove 5000 strings
• Remove 5000 sorted sets

Test Execution

• Windows azure - 14 day continuous run
  • 1-4-4 setup
  • All versions
• Local machines - 1 day continuous run
  • All setups
  • All versions
• Local machines - 16 days continuous run
  • 1-4-4 setup
  • 2.6-32
  • 2.6-64

Redis.conf

Redis configuration file used for both Master and Replicas can be found here

Bugs found

• Server crash due to closing empty handle
• Wrong console memory reporting (initially assumed as memory leak).