Load Testing on the On-Premises API

Overview

Businesses may want to request a Load Testing session to determine how their On-Premises deployment is going to perform under a specific load. Similarly, they can measure how much traffic their webhook application is able to handle by simulating scenarios that are similar to what they would expect in production.

With this information, businesses should have enough data to understand what improvements are necessary in their On-Premises deployment as well as in their webhook application to accommodate their business needs and cope with their volume demand.

Introduction to Load Test on On-Premises API

What is a Load Test?

Load testing is a common engineering practice used to evaluate your system performance under a certain load in a controlled environment. Performance evaluation is available for all premium Business Solution Providers. These tests can help evaluate the message throughput of your system under production-like traffic before going into production and can help measure the resilience of your webhook system. It is commonly used to:

• Troubleshoot performance issues
• Benchmark system performance
• Optimize infra cost

What will we be testing?

Load testing will enable you to understand how your On-Premises setup will behave under a specific load or traffic. You'll learn how many inbound and outbound messages your setup is capable of handling.

Additionally you'll learn how your webhook application is going to perform under that traffic as well. For each outbound message you successfully send from your On-Premises setup, you're expected to receive three webhook notifications i.e. one for each messaging event: message sent, message delivered and message read. You'll also get one webhook notification for each inbound message.

Finally, load testing can inform you on how each component of your On-Premises setup will perform during a live campaign. You'll be able to identify bottlenecks in terms of memory pressure, CPU used, database and network latencies etc

What types of Load Tests are available?

WABiz: Request Inbound Load Testing (beta) to test inbound loads. During each test, messages are sent to the phone number you want to load test at the requested throughput for the requested duration. For each phone number you want to load test, make sure that:

• The phone number under test is not used by real users during the test duration
• The phone number under test is a bot that replies to incoming messages (you can define what type of messages you want to reply with based on your needs)
• The Grafana monitoring dashboard is set up for the phone number under test according to our dev docs

WABiz: Request Outbound Load Testing (beta) to test outbound loads. During each test, messages are sent from the load test number to real numbers by mentioning throughput. To initiate a load test, you need to:

• Prepare a messaging script that will be executed during the load test. This script will send a message to at least 250k phone numbers that can be executed at different throughputs using a message template provided by Meta. Both you and Meta will monitor the execution of the script during the test
• Create a new setup that can support high throughput and register an outbound load test number provided by Meta. Note that, with the test phone number provided by Meta, no messages will be delivered and no charges will be incurred however you will receive the sent, delivered, and read webhook notifications
• Set up a Grafana monitoring dashboard for the high throughput template according to our dev docs

Outbound Load Testing on On-Premises

Preparing for the Load Test

To prepare for the Load Test session, businesses should have their On-Premises deployment ready and set up with the correct number of shards according to their desired throughput. Although the business' On-Premises setup will be used, a test phone number will be provided by Meta along with its certificate for registration. Please check the Number Preparation section below for more information.

Grafana monitoring is mandatory for On-Premises outbound load testing, so it should be in place as well. To correctly set up Grafana monitoring, please refer to the documentation.

Additionally, businesses may already have their own load test script which can be specific to their needs. However if they don't have that, Meta can provide one. Please check the Load Test Script section below for more details.

Finally, businesses should create a template that will be used for sending out the messages as part of the load test. Meta will copy this template to the test WABA that will be used during the load testing session. Please check the Number Preparation section below for more information.

Number Preparation

For the purpose of conducting outbound load testing on On-Premises setups, Meta will furnish a dedicated test phone number. This particular phone number is specifically configured to refrain from delivering messages to end users. When messages are dispatched via the test phone number, they will not reach their intended recipients but will nonetheless activate the expected webhook notifications. Consequently, the webhook application can appropriately respond to each event.

It is crucial to bear in mind that the test phone number should not receive any incoming messages. Its sole purpose is to emulate outbound message interactions. If a message is directed to the test phone number, a protective mechanism will be triggered, leading to its disconnection.

On the day of the load testing session, Meta will provide the business with the test phone number as well as its certificate so that the business can register it on their On-Premises deployment.

In addition to the test phone number, Meta will supply a comprehensive roster of 250k fictitious phone numbers that serve as the target for the outbound messages. This list enables us to simulate a substantial audience, contributing to the efficacy of the outbound load test.

Lastly, the business needs to provide a template that they wish to use during load testing. This includes the WABA under which that template lives. With this information, Meta will be able to copy it over to the test WABA, so it can be used during the load testing session.

Load Test Script

Businesses may have their own custom load test script that covers their business scenarios. If they don't have that, they can use the load test script provided by Meta.

The load test script created by Meta is available for download.

This project is created using Node.JS and a load test library called Artillery. You'll find a README.pdf document within the project files, which contains information on how to set up Artillery as well as how to run the load test script.

You'll also find a JSON file with 250k fake phone numbers that are supposed to be used to send out messages as part of the load test.

Important Files and Parameters

• config.json - Use this file to add the access token that the script will use in order to submit requests to the On-Premises API. More information on how to get an access token is available in the dev docs.
• config.yml - In this file, you'll need to provide the base URL for the Web App in the target parameter. Under phases, you can tweak the duration and arrivalRate parameters according to your needs.

  • duration - indicates the number of seconds that requests will be sent to the On-Premises API. This is the actual duration of the load test.
  • arrivalRate - indicates the number of requests that will be sent to the On-Premises API each second. This is the number of messages per second.
• template.json - In this file you'll find the actual payload for the send message request. This payload represents a template message, which has to match the template name and parameters that the business informed as part of the load testing session.

How to run the load test script

To execute the load test script, it is essential to have Node.js installed. Please ensure that you have downloaded and installed Node.js before proceeding with the following guidelines:

1. Download the load test script.
2. Unpack the package contents into a designated folder.
3. Launch the terminal and navigate to the directory where the script has been extracted.
4. Modify the parameters as required, following the instructions provided in the preceding section.
5. Install Artillery by executing the following command:

   npm i artillery@latest

6. To initiate the load test, execute the command:

   artillery run config.yml

Inbound Load Testing on On-Premises

For inbound load testing, Meta will be responsible for sending the desired volume of messages to the phone number under test. The deployment where the phone number under test lives must have an application capable of receiving webhook notifications, and this application should be able to acknowledge that it received the request by responding with HTTP 200.

To monitor the overall state of the deployment during the inbound load test, Grafana must also be correctly set up according to our dev docs.

Mixed Load Testing on On-Premises

Similar to the inbound scenario, the mixed load testing will also rely on Meta to send the incoming volume of messages to the phone number under test. What changes here is that the webhook application is expected to reply to the incoming messages according to the scenarios the business wants to test. As part of the inbound load test, Meta will whitelist the phone number under test so that no charges will occur when the application behind the webhook replies to the incoming messages.