Introduction
Edge computing is ideally suited for querying populations of fast data streams. However, developing and evaluating edge-based query processing algorithms is challenging due to the decentralized setting and the lack of effective tools. For example, current simulators fail to execute real queries. To rectify these shortcomings, we propose TEQ, a Testbed for Edge-based Query processing algorithms, designed expressly to be open, enabling extensible simulation and algorithm deployment, and to be developer-friendly, simplifying otherwise cumbersome simulation, complex algorithm management, and tedious evaluation.
TEQ encompasses novel techniques to achieve these capabilities, including:
- a streamlined framework for implementing decentralized algorithms as modular, Docker-based executables;
- data and query playback controls to enable reproducible and consistent evaluations;
- mechanisms for transforming runtime metrics into system-wide metrics desired by algorithm developers.
Currently in the open-source process, TEQ aims to foster community contributions and drive innovation in edge-based query processing.
Run TEQ
Since TEQ uses some Linux kernel tools, TEQ is currently only supported on Linux. On Windows system, you can use WSL to run TEQ.
In addition, the TEQ needs the following environments:
- Docker, and the image required by TEQ (the image is automatically downloaded from the image repository when it is run for the first time)
- tc (Traffic Control) for network model. This is part of the Linux kernel and is usually already installed.
These environments are usually very easy to install.
Once you have ensured that your environment meets the requirements, you can clone a copy directly from our repository, and in the ./src/example folder, you can find an example of using out testbed.
in this folder, we have a main class Main.java that you can run directly. This class will start the simulator and run the example code.
Implement Your Own Algorithm
To implement your own algorithm, you need to create a Simulator object to operate the entire simulator first. The Simulator requires a Network class as the networkHost node. If no additional operations are needed, you can directly use the default constructor.
The simulator by default contains four layers, and for each layer, you need to implement an abstract class to describe the data processing logic at each layer in edge computing. In the example, we implemented a local STkFIQ model based on Flink. You can read the code in the example folder (which is very easy to understand) to learn about the TEQ programming process.
After adding all the layer needed to the simulator, finally, you can run the simulator by calling the start method of the Simulator object. The simulator will start the simulation process and run the algorithm you implemented.