Category: ABAP

A significant portion of ABAP programming is selecting data from database tables and reconstructing them for specialized use cases. Information shown to the consumer of the program is temporary fields, calculated using persisted data as input. Since the development of the HANA database, SAP changed the approach to accessing and calculating data in their programs and advises developers to do so as well. Most notably, the “Code to Data” paradigm aims to move unnecessary data retrieval and calculations from the application server to the database server. Calculated fields – be it through arithmetic calculations, string manipulations or complex case expressions – can now be provided without even accessing the ABAP layer. This is still a relatively new concept to many developers, as most of the code they have written so far juggles the retrieved data in ABAP code. For maximum time efficiency, new methods must be learned, some of which I will briefly explain. This blog does not contain an introduction with examples of either method, but merely describes the function and its cases of use.

CDS Views Core

Data Services (CDS) is the key component of all data modeling in HANA. While widely known, some developers still treat it as a mere join on database tables, unaware that CDS is a fully-fledged SQL language, that is also relatively easy to learn. Anything previously done in Open SQL can be directly implemented in the view without the use of any ABAP. It also provides annotations a consumer can utilize. Multistep operations might require the use of several CDS views, creating a view stack that can become hard to read.

ABAP Code

In special cases, it might still be necessary to use ABAP code to retrieve and calculate data. In fact, some of the new HANA tools are made specifically for ABAP.
CDS custom entities are data definitions that do not use SQL. Unlike CDS views, they have no data source but gather the data from ABAP code (which, of course, can include data sources). It requires a dedicated class that implements the query in detail, including counting and paging. Custom entities are primarily used when SQL is incapable of retrieving the requested data, for example, from a remote data source that can only be accessed by ABAP code or when the business logic is simply too complex for SQL. The finished custom entity can then be used like any other data source. It is like a temporary data view that is calculated in the ABAP code.
In the context of RAP, CDS views can make use of virtual fields that are calculated in dedicated ABAP classes. The regular fields of the view are used as input to calculate the virtual field for each row individually. A simple example would be an adjusted price to a base price that is fetched from the database. In contrast to customs entities, they are only single fields of a view, not an entire data source.

SQLScript Code

For cases when CDS views are not sufficient to push down ABAP code to the HANA database, methods to directly program on it have been developed. The HANA database has its own version of SQL, SQLScript, which is considerably closer to a full programming language than ABAP SQL. Database developers that create HANA views are already familiar with this language, but the ABAP language also provides access to it.
Special methods called ABAP-managed database procedures (AMDP) allow the use of SQLScript code. It is written on the application layer but executed on the database layer. One dedicated use of AMDPs is table functions. They fetch and calculate data on the database through SQLScript and provide the result as a temporary table. In contrast to customs entities, all of this happens on the database. It provides more capabilities than CDS and Open SQL and even allows the use of special HANA tools like machine learning libraries. If optimal performance is desired, AMDPs should be preferred over the ABAP code. Of course, it requires learning a new language, which has a lot more depth to it than other SQL languages.

Parallelization of programs is typically not a problem ABAP developers encounter frequently. In most cases, database access via OpenSQL has a much greater impact on report performance than processing and outputting the data. Nevertheless, there are scenarios where parallelization can dramatically speed up processing. To demonstrate the fundamental workings of parallelization, I chose the visualization of the Mandelbrot fractal. The Mandelbrot fractal is probably the best-known example of its kind and an excellent example of a process that can be parallelized. But first, the basics need to be clarified: What exactly is parallelization, and what prerequisites must a process meet to benefit from parallelization?

Serial and Parallel Processes

Typically, ABAP programs and reports are processed serially. This means the program is executed line by line sequentially. A typical report reads parameters from the selection screen, selects corresponding data from the database, processes it, and outputs it. Loops are often used to handle the processing of many elements. The program processes all database entries one by one, as shown in the following diagram::

Loops are a good indicator that processing steps could potentially be parallelized. The prerequisite is that the process steps are independent of each other, meaning one loop iteration does not affect the result of another. This can be enforced, for example, by using separate tables for the data read from the database and the results, with a 1:1 assignment. The flow of a parallelized report might look like this:

Since the processing steps illustrated in the diagram occur simultaneously, the program runtime is dramatically reduced. Program parts that run in parallel are also called threads. Unfortunately, not all processes can be parallelized equally well. Designing a program flow that can be processed in parallel is a science in itself, but here are some examples of processes that are more or less suitable for parallelization:

ABAP-Threads

The next question is how to parallelize processes in ABAP. The programming language itself does not provide simple syntax to describe parallel execution of program parts. This is where the ABAP-Threads framework comes into play. It enables parallel processing through a mechanism that starts a new dialog process in the background for each thread. This means the maximum number of concurrently running threads depends on the maximum number of dialog processes available in the system. If too many threads are started, there may not be enough dialog processes left for other users. This must be kept in mind. If in doubt, the parameter rdisp/wp_no_dia in transaction RZ11 can be checked. It contains information about the maximum number of dialog processes in the system. Threads always use a dialog process, even if the report itself is started as a batch process in the background.

ABAP-Threads can be imported into the system using ABAP-Git. Here is the link to the GitHub repository: ABAP-Threads

The programming concept of ABAP-Threads is based on so-called workers. For each thread, a worker object must be created. This object is an instance of a worker class in which the parallel program code is implemented. The worker class inherits from the abstract class ZCL_THREAD_RUNNER_SLUG and has the following core components:

• A single attribute (member variable) where all parameters required for the subtask and the results are stored. This serves as the data interface for the worker and should use a structure as its data type.
• The method CREATE, which creates the worker instance and copies the parameters into the attribute. (The constructor cannot be used for technical reasons.)
• The method GET_STATE_REF, which returns the reference to the attribute.
• The method RUN, which contains the program code to be executed in parallel.

The attribute of the worker class serves as the sole interface for data exchange between the main program and the worker. This also implicitly ensures that there is no communication between worker instances. This restriction helps avoid common pitfalls in parallel programming, such as race conditions or deadlocks. However, it also means that the partial results of the workers may need to be copied back into a local variable of the main program after processing, which involves some overhead.

The program flow for parallel processing using ABAP-Threads typically looks like the diagram below:

Implementation Using the Mandelbrot Fractal Example

Parallelization Strategy

To parallelize a process, a strategy must be developed to split it into many smaller subprocesses. First, the dependencies in the process need to be examined. For instance, it would not be possible to calculate net salary before gross salary because the net salary depends on the gross amount. In the case of the Mandelbrot fractal, each pixel in the image is practically independent of the others, providing a lot of flexibility to distribute the work among almost any number of workers. In this example, the image is split horizontally and assigned to the workers.

The image data is generated using a helper class capable of producing a file in bitmap format. The color of each pixel can be directly controlled and described in binary format or as HSV (hue, saturation, value). There are numerous resources online regarding the algorithm for generating the Mandelbrot fractal.

The Worker Class

The attribute of the worker class uses the structure t_data_and_parameters, which includes both the parameters necessary for image generation and the image data for the horizontal segment of the image. The definition is located in the public section, as it is also needed outside the class.

The attribute itself should then be defined in the private section:

The CREATE and GET_STATE_REF methods are redefined and implemented according to the schema. In the redefinition of the RUN method, the algorithm is implemented to generate the image segment based on the parameters in the attribute and save it back into the attribute. The image rows are iterated over, and the color of each pixel is determined using a Mandelbrot helper class.

The Report

After implementing the worker class, it can be used in a report. First, worker instances are created using a loop. They can then be started with the RUN_PARALLEL method. To simplify later access, the references to the workers are stored in a table. In the code snippet, the worker class is called LCL_PARALLEL_RENDERER.

The program then waits until all workers are finished. A worker completes its task once its RUN method exits. This can be checked using the IS_READY method.

Subsequently, the partial results can be combined into a complete image. The result is a full representation of the Mandelbrot fractal displayed as a bitmap in the SAP GUI:

Performance

The most interesting part: How much performance improvement can be achieved through parallelization? To answer this, multiple benchmarks were conducted on a test system. For each thread count, five measurements were taken, and the average was calculated. The system runs on a virtual machine with 24 vCores. The number of dialog processes is limited to 8, with one reserved for the report itself.
The results are shown in the following two diagrams. The standard deviation of the measurements is negligible and is therefore not included in the representation.

It is evident that processing speed increases with parallelization. However, the relationship between the number of threads and performance improvement is not linear. This is due to the potential for optimization in the parallelization strategy. The naive approach of assigning an equal number of image rows to each worker is suboptimal because the algorithm for generating the image is recursive. Image areas containing black pixels require significantly longer calculation times than the rest. The bottleneck is the worker that takes the longest. For this reason, processing speed increases beyond seven workers. Even though only seven workers can run simultaneously, threads that finish their image segment early can take over the next part. The overhead from creating worker instances and copying image data is negligible in this example.

Conclusion

In conclusion, parallelization using ABAP-Threads is an effective way to significantly improve process execution speed. However, the performance gain depends heavily on the parallelization strategy. Additionally, knowledge of system hardware and configuration of the maximum number of dialog processes is essential to avoid conflicts with other system users.
Parallelization is primarily useful for offline scenarios when the system load is low. For instance, updates or time-critical processes can be accelerated when they would otherwise not fit within the available timeframe. In a productive system’s normal operation, it can block other processes and reduce available computational power for other users, so parallelization should be used cautiously.