ABOT : TEST ORCHESTRATION SOLUTION

Introduction of ABot Application and service complexity demands intricate setup, configuration, and test scripts. For successful and faster roll-outs of network services like Mobility and IoT on different platforms, it is necessary to define call flows and execute them on multiple setups in a CI/CD environment.

The ABot Test Orchestration Solution uses the concept of BDD to define and execute test scenarios enabling collaboration amongst multiple stakeholders. The test cases are written in a user-friendly Domain Specific Language (DSL) for Product Managers, Dev/QA engineers, Solution Architects as well as Field personnel to define and reuse the test cases. ABot can be seamlessly invoked from different CI engines like Jenkins for scheduling the test execution. The framework integrates with orchestrators like Juju and Cloudify for Continuous Deployment (CD) and lifecycle management on NFV platforms. ABot can incorporate plugins to emulate components of complex systems like Mobility and IoT for comprehensive functional and performance testing.

ABot comes integrated with an intuitive web-based UI that provides a graphical summary, detailed review of the test results and test analytics. The test results are viewable through the ABot UI as well as through the Jenkins dashboard; thereby enabling easy certification of the System under Test (SUT).

Look out for details on the 3 main applications for ABot:

  • 1. ABot: A Test Orchestration Framework
  • 2. ABot: A Test Harness emulating network components
  • 3. ABot: Solution for VNF Certification and Deployment Validation

The ABot solution includes a Test Orchestration framework that enables Service Providers and OEMs to define reusable tests based on a user-friendly Domain Specific Language (DSL) and schedule their execution through any CI engine like Jenkins.

ABot uses the concept of BDD (Behaviour driven development) to define service-level test scenarios in a high-level language using spec-driven acceptance criteria thereby enabling collaboration with multiple stakeholders. Since test cases are written in English like language (no programming syntax), even a field engineer can easily add a field level test case there by enhancing the test repository. These feature files are mapped to relevant specifications, call flows and procedures, hence they can be provided as a part of RFP response for easy customer verification.

ABot provides a repository for user-defined test cases as well as the test bed configuration. These test cases are in the form of feature files and can be executed either individually or in groups through CI systems like Jenkins. The test results are viewable through the ABot UI as well as through the Jenkins dashboard; thereby enabling easy certification of the System under Test (SUT).

ABot comes integrated with an intuitive web-based UI that provides a graphical summary of the test results; primarily focusing on the aggregate/holistic view. One can then drill down to detailed results indicating the status of each step in a test case along with relevant artifacts such as log files and PCAP captures.

Managing complex mobility services like IoT, M2M & AI requires seamless interoperability between different network functions. Executing reusable DSL-based feature files across multiple environments helps uncover specific problems as well as performance constraints on different physical and virtual platforms. The ABot supported plugins can change parameters for different call flows with ease and yet support complex use cases under demanding load conditions as emulated by advanced traffic generators like Ixia and dsTest.

ABot as a test harness can emulate complex telecom network systems like the vEPCvIMS as well as vIoT under a single umbrella offering. If the need of the user is to simulate different network components like that of a vEPC system, namely – eNodeB, MME and SPGW, then the user is given the ability to do so. The idea is to emulate important features of the system and provide the capability of performing functional and load testing on all of its features.

ABot comprises of NFV templates for feature-level call flows which provide an easy interface to modify the parameters of a single call flow. Vendors can send these templates with the RFP specifications for visibility and validation. The customers can not only evaluate the DSL based test case but also modify values to test the threshold. While aiming to support a wide spectrum of protocol stacks in the mobility domain, ABot also supports wireless stacks built for the short range IoT world, such as Wifi, Bluetooth, Bluetooth Mesh and ZigBee.

ABot helps test granular pieces of the system in pre-deployment and lab environments. The ABot supported plugins test protocol stacks like S1APGTPv2CGTPv1UDiameterRadiusSIP, and RTP. ABot has the ability to support different releases of specification standards such as 3GPP. New feature call flows can be implemented in minimal time using the existing template files. ABot can answer questions such as, does my implementation of S1 Handoff support the information elements of 3GPP Release 14?

ABot targets the need for emulation while testing. Apart from its low footprint and lab-oriented approach to traffic generation; ABot also supports plugins for industry standard traffic generators like Ixia and dsTest used at large environments with production-grade requirements. Whether your needs are for lab’s or production environment’s trouble shooting capability, ABot provides the necessary plugins like CI enginesprotocol stack emulators, orchestrator and 3rd party tools. Tailor made plugins can also be integrated with the ABot framework.

As cloud computing and NFV becomes a reality, VNF deployment verification poses a major challenge. There has been a fair amount of focus on VNF on-boarding but the industry is yet to find an automated validation mechanism for virtualized network services deployed at multiple locations. Operators need automated deployment of test harnesses on NFV-compliant infrastructure with scripted execution of well-defined test on the target VNFs and the ability to troubleshoot issues using easily understood results. 

ABot is easily deployed on any virtualized platform and performs functional and performance validation on the target VNFs. It has plugins to integrate with test bench components (such as 3 rd  party traffic generators and metrics collectors) and can run repeated test scenarios. Abot has the capability to perform system tests on multi-vendor VNFs for service-level validation like VoLTE. It can also perform VNF lifecycle management verification as well as validation of self-healing and auto-scaling policies of the deployed VNFs. ABot can certify VNFs deployed as virtual machines on cloud platforms (such as OpenStack and VNFs deployed as containers and managed by Kubernetes.

The ABot solution can be deployed along with selected NFV orchestrator on any target cloud platform. It can trigger VNF deployment using the NFVO/VNFM and subsequently invoke functional and load tests for VNF certification and deployment validation. ABot supports multiple NFVO/VNFM combinations such as Cloudify, Juju and OpenBaton.

ABot uses the plugins for emulation capabilities to execute functional and performance tests on the deployed VNFs using templates. ABot enables easy verification against different KPIs for each test scenario as well as presentation of all statistical values in the generated reports.

ABot Application Scenarios

Watch a video demo of the Joint Solution by Ixia and Rebaca which aims to simplify NFV Deployment Testing.

Features and Specification

Supports testing of virtual, hybrid and proprietary hardware based wireless network

on multiple cloud platforms using different NFV orchestrators

Read More

Performance and Load Testing

Read More

EPC, IMS, VoLTE

Conformance and Technical Specifications

Read More

Easily extensible protocol adapters SDK

for supporting multiple Network Function

Read More

Advanced Reporting

based on Predictive Test analytics for feature testing, regression testing, integration testing, performance testing etc.

Read More

Available in Canonical Juju charm store

for deployment in multiple cloud platforms including Openstack

Read More

The VoLTE use case is the traditional system integrator’s nightmare. In order to accurately measure the KPIs, ABot provides the capability of simulation of key nodes, data collection and analysis of results. Test cases can be augmented and modified from existing library. Test cases range from Signaling Validation, Multiple Codec Characteristics, Voice QoS and QoE(MOS) – under different reference conditions as well as the capability of doing comparative studies under these conditions. The reference conditions refer to the variations in mobility, network congestion, multi-vendor characteristics. Benchmark testing and comparison with OTT & VoIP services are also covered for the ability to make business decisions.


Sample Feature File for VoLTE

The LTE Access network is the single biggest advancement toward high speed data mobility. The use case ranges from the traditional voice, video, text but with data and variable QoS, with greater mobility and higher quality. ABot contributes to these advancements by providing a means to test the LTE access network in the following areas for performance benchmarking and functionality: UE Attach/Detach, Idle Mode, Authentication, Security Mode, EMM and ESM Procedures, Handover and Tracking Area Procedures. The focus of the test cases are in being able to measure the capability of the network in terms of services and the resilience of network when challenged by load and mobility requirements.


Sample Feature File for EPC (Authentication Failure)

The IMS network allows for the convergence of different kinds of networks. The ABot Test cases allow for the testing of various aspects of the IMS networks – Registration, Authentication, Call Control, Service Discovery and Triggers, Messaging, Application Servers, Third Party Call Control and Conferencing. The focus of the test cases lies in testing of the network from the point of view of convergence and resilience of the same network, when challenged by load.


Sample Feature Files for IMS